Question about bytecode stability
I'm getting ready to get back into an old personal project with some new ideas. Before I do, I need to ask a question regarding the stability of CPython bytecode. Before you say it, I am well aware and fully understand that the opcodes may change in value and semantics between 3.x and 3.(x+1) without notice, and that's acceptable to me. My question is, are the opcodes guaranteed stable across the lifetime of a single 3.x release? In other words, are they guaranteed to not change values or semantics between 3.x.y and 3.x.(y+1)? Reading through the list of opcodes in the dis documentation, it seems that all changes have occurred in 3.x.0, so it seems the answer would be yes, but then the "CPython implementation detail" paragraph at the top doesn't specify that and is a little vague on whether that's true or not. (For context, my personal project is a BASIC implementation in Python that would compile to Python bytecode.)
On Mon, Apr 6, 2020 at 1:53 PM Jonathan Goble <jcgoble3@gmail.com> wrote:
I'm getting ready to get back into an old personal project with some new ideas. Before I do, I need to ask a question regarding the stability of CPython bytecode. Before you say it, I am well aware and fully understand that the opcodes may change in value and semantics between 3.x and 3.(x+1) without notice, and that's acceptable to me.
My question is, are the opcodes guaranteed stable across the lifetime of a single 3.x release? In other words, are they guaranteed to not change values or semantics between 3.x.y and 3.x.(y+1)? Reading through the list of opcodes in the dis documentation, it seems that all changes have occurred in 3.x.0, so it seems the answer would be yes, but then the "CPython implementation detail" paragraph at the top doesn't specify that and is a little vague on whether that's true or not.
The best way to look at this is to consider how long a .pyc file is valid. They're currently named something like __pycache__/modulename.cpython-38.pyc which is a fairly clear indication that the cached compiled module should be valid for any CPython 3.8.x release. So yes, you should be safe within any given release. The reason it's a CPython implementation detail is also hinted at in the file name: if you were to use Jython or PyPy or IronPython or MicroPython or any other implementation, you can't be sure your bytecodes will work, even if the version number is the same. (An interpreter can be compatible with Python 3.8 without having the same bytecode as CPython 3.8.) So for your purposes, it sounds like, yes, this WILL work. :) ChrisA
On Mon, Apr 6, 2020 at 12:02 AM Chris Angelico <rosuav@gmail.com> wrote:
On Mon, Apr 6, 2020 at 1:53 PM Jonathan Goble <jcgoble3@gmail.com> wrote:
I'm getting ready to get back into an old personal project with some new
ideas. Before I do, I need to ask a question regarding the stability of CPython bytecode. Before you say it, I am well aware and fully understand that the opcodes may change in value and semantics between 3.x and 3.(x+1) without notice, and that's acceptable to me.
My question is, are the opcodes guaranteed stable across the lifetime of
a single 3.x release? In other words, are they guaranteed to not change values or semantics between 3.x.y and 3.x.(y+1)? Reading through the list of opcodes in the dis documentation, it seems that all changes have occurred in 3.x.0, so it seems the answer would be yes, but then the "CPython implementation detail" paragraph at the top doesn't specify that and is a little vague on whether that's true or not.
The best way to look at this is to consider how long a .pyc file is valid. They're currently named something like __pycache__/modulename.cpython-38.pyc which is a fairly clear indication that the cached compiled module should be valid for any CPython 3.8.x release. So yes, you should be safe within any given release. The reason it's a CPython implementation detail is also hinted at in the file name: if you were to use Jython or PyPy or IronPython or MicroPython or any other implementation, you can't be sure your bytecodes will work, even if the version number is the same. (An interpreter can be compatible with Python 3.8 without having the same bytecode as CPython 3.8.)
So for your purposes, it sounds like, yes, this WILL work. :)
ChrisA
Thanks, I never really looked closely at .pyc files, so I missed that. Good to know that I only have to watch for changes between feature releases. I knew that about "CPython implementation detail", that it wouldn't be valid on non-CPython implementations, but didn't clearly state so in the OP. Regardless, I appreciate the reminder.
The best way to look at this is to consider how long a .pyc file is valid. They're currently named something like __pycache__/modulename.cpython-38.pyc which is a fairly clear indication that the cached compiled module should be valid for any CPython 3.8.x release.
Perhaps an even better indicator is the “magic number” that invalidates the cached bytecode. This can change several times during the alpha and beta phases, and a (mostly complete) record of changes can be found here: https://github.com/python/cpython/blob/master/Lib/importlib/_bootstrap_exter...
On Mon, 6 Apr 2020, 06:30 Brandt Bucher, <brandtbucher@gmail.com> wrote:
The best way to look at this is to consider how long a .pyc file is valid. They're currently named something like __pycache__/modulename.cpython-38.pyc which is a fairly clear indication that the cached compiled module should be valid for any CPython 3.8.x release.
Perhaps an even better indicator is the “magic number” that invalidates the cached bytecode.
Probably the best indicator is PEP 6, which explicitly states bytecode compatibility is a requirement: "All .pyc and .pyo files must work (no regeneration needed) with all bugfix releases forked off from a major release." https://www.python.org/dev/peps/pep-0006/#prohibitions --
06.04.20 06:48, Jonathan Goble пише:
My question is, are the opcodes guaranteed stable across the lifetime of a single 3.x release? In other words, are they guaranteed to not change values or semantics between 3.x.y and 3.x.(y+1)? Reading through the list of opcodes in the dis documentation, it seems that all changes have occurred in 3.x.0, so it seems the answer would be yes, but then the "CPython implementation detail" paragraph at the top doesn't specify that and is a little vague on whether that's true or not.
At least once it was changed in the bugfix release (3.5.2). It was minor change, and mostly compatible, but it changed the magic number, so all pyc files were needed to regenerate, and *that* caused problems for distributors. We no longer want to repeat this. But I do not know what we will do if a fatal flaw in the bytecode be found after release.
Thanks for the feedback, everyone. I'm satisfied that the bytecodes are stable enough to work for my purpose. Now I just have to find time to work on it around my schoolwork.
participants (5)
-
Brandt Bucher -
Chris Angelico -
Jonathan Goble -
Serhiy Storchaka -
Thomas Wouters