[Python-Dev] Mysterious Python pyc file corruption problems

[Barry Warsaw]

On May 17, 2013, at 03:02 PM, Terry Jan Reedy wrote:

>What I mean is, is the corruption (not the detection of corruption) only
>happening during mass compilation of the stdlib? When user imports a single
>non-stdlib file he has written the first time, does that ever get corrupted.

It's not limited to the stdlib, but yes, as far as we can tell, it happens
during package installation on an end-user system, when the trigger script
mass byte-compiles a package's Python source files.

>I intended to suggest that py_compile be changed to do that. Then Brett said
>it already had for 3.4. I see no reason why not to backport, but maybe
>someone else will.

I tend to agree.

>The main design use of marshal is to produce .pyc files that consist of a
>prefix and marshalled codeobject. Perhaps marshal.dump(s) should be extended
>to take a third prefix='' parameter that would be prepended to the result as
>produced today. .dump first does .dumps, though inline. I assume that .dumps
>constructs a string by starting with [], appending pieces, and joining. At
>least, any composite object dump would. The change would amount to starting
>with [prefix] instead of []. Then py_compile would amount to
>   pyc = <pyc-prefix>
>   marshal.dump(codeobject, file, pyc)

That wouldn't be a backportable change, and right now I'm trying to stay
focused on fixing this specific problem. ;)

-Barry