PEP 3147 - new .pyc format

[Steven D'Aprano]

On Sun, 31 Jan 2010 04:44:18 +0100, Alf P. Steinbach wrote:

>> The relationship between byte code magic number and release version
>> number is not one-to-one. We could have, for the sake of the argument,
>> releases 3.2.3 through 3.5.0 (say) all having the same byte codes. What
>> version number should the .pyc file show?
> 
> I don't know enough about Python yet to comment on your question, but,
> just an idea: how about a human readable filename /with/ some bytecode
> version id (that added id could be the magic number)?

Sorry, that still doesn't work. Consider the hypothetical given above. 
For simplicity, I'm going to drop the micro point versions, so let's say 
that releases 3.2 through 3.5 all use the same byte-code. (In reality, 
you'll be likely looking at version numbers like 3.2.1 rather than just 
3.2.) Now suppose you have 3.2 and 3.5 both installed, and you look 
inside your $PYTHONPATH and see this:

spam.py
spam.pyr/
    3.2-f2e70a0d.pyc


It would be fairly easy to have the import machinery clever enough to 
ignore the version number prefix, so that Python 3.5 correctly uses 
3.2-f2e70a0d.pyc. That part is easy.

(I trust nobody is going to suggest that Python should create multiple 
redundant, identical, copies of the .pyc files. That would be just lame.)

But now consider the human reader. You want human-readable file names for 
the benefit of the human reader. How is the human reader supposed to know 
that Python 3.5 is going to use 3.2-f2e70a0d.pyc? 

Suppose I'm running Python 3.5, and have a troubling bug, and I think "I 
know, maybe there's some sort of problem with the compiled byte code, I 
should delete it". I go looking for spam.pyr/3.5-*.pyc and don't find 
anything. 

Now I have two obvious choices: 

(1) Start worrying about why Python 3.5 isn't writing .pyc files. Is my 
installation broken? Have I set the PYTHONDONTWRITEBYTECODE environment 
variable? Do I not have write permission to the folder? WTF is going on? 
Confusion will reign.

(2) Learn that the 3.2- prefix is meaningless, and ignore it.

Since you have to ignore the version number prefix anyway, let's not lay 
a trap for people by putting it there. You need to know the magic number 
to do anything sensible with the .pyc files other than delete them, so 
let's not pretend otherwise.

If you don't wish to spend time looking up the magic number, the solution 
is simple: hit it with a sledgehammer. Why do you care *which* 
specific .pyc file is being used anyway?

rm -r spam.pyr/

or for the paranoid:

rm spam.pyr/*.py[co]

(Obviously there will be people who care about the specific .pyc file 
being deleted. Those people can't use a sledgehammer, they need one of 
those little tack-hammers with the narrow head. But 99% of the time, a 
sledgehammer is fine.)

Frankly, unless you're a core developer or you're hacking byte-code, you 
nearly always won't care about the .pyc files. You want the compiler to 
manage them. And for the few times you do care, it isn't hard to find out:

>>> import binascii, imp
>>> binascii.hexlify(imp.get_magic())
b'4f0c0d0a'

Stick that in your "snippets" folder (you do have a snippets folder, 
don't you?) and, as they say Down Under, "She'll be right mate".


-- 
Steven