[Python-ideas] Add more information in the header of pyc files

[Antoine Pitrou]

On Tue, 10 Apr 2018 18:49:36 +0300
Serhiy Storchaka <storchaka at gmail.com>
wrote:
> 
> 1. More stable file signature. Currently the magic number is changed in 
> every feature release. Only the third and the forth bytes are stable 
> (b'\r\n'), the first bytes are changed non-predicable. The 'py' launcher 
> and third-party software like the 'file' command should support the list 
> of magic numbers for all existing Python releases, and they can't detect 
> pyc file for future versions. There is also a chance the pyc file 
> signature will match the signature of other file type by accident. It 
> would be better if the first 4 bytes of pyc files be same for all Python 
> versions (or at least for all Python versions with the same major number).

+1.

> 2. Include the Python version. Currently the 'py' launcher needs to 
> support the table that maps magic numbers to Python version. It can 
> recognize only Python versions released before building the launcher. If 
> the two major numbers of Python version be included in the version, it 
> would not need such table.

+1.

> 3. The number of compatible subversion. Currently the interpreter 
> supports only a single magic number. If the updated version of the 
> compiler produces more optimal or more correct but compatible bytecode 
> (like ), there is no way to say that the new bytecode is preferable, but 
> the old bytecode can be used too. Changing the magic number causes 
> invalidating all pyc files compiled by the old compiler (see [4] for the 
> example of problems caused by this). The header could contain two magic 
> numbers: the major magic number should be bumped for incompatible 
> changes, the minor magic number should be reset to 0 when the major 
> magic number is bumped, and should be bumped when the compiler become 
> producing different but compatible bytecode.

-1.  This is a risky move (and costly, in maintenance terms).  It's easy
to overlook subtle differencies that may translate into
incompatibilities in some production uses.  The rule « one Python
feature release == one bytecode version » is easy to remember and
understand, and is generally very well accepted.

Regards

Antoine.