[Python-Dev] Mysterious Python pyc file corruption problems

[Terry Jan Reedy]

On 5/16/2013 2:04 PM, Barry Warsaw wrote:

> No, it's all different kinds of machines, at different times, on different
> files.  So far, there's no rhyme or reason to the corruptions that I can
> tell.

If the corruption only happens on Ubuntu, that would constitute 'rhyme' 
;-). I realize that asking for reports on other systems is part of the 
reason you posted, but I don't remember seeing any others yet.

> We're trying to instrument things to collect more data when these
> failures do occur.

Do failures only occur during compileall process? (or whatever 
substitute you use).
At the end of py_compile.complile, after the with block that opens, 
writes, flushes, and closes, you could add
   with open(cfile, 'rb') as fc: <read header and unmarshal rest>
This would be a high-level write and verify.

Verify would be a bit faster if marshal.dump were replaced by 
marshal.dumps + write to keep alive the string version of the code 
object. Then the codeobject comparison in the verify step would be 
replaced by string comparison.

You could also read and verify (by unmarshal) after the compile-all 
process (faster than importing).

Terry