[Python-Dev] Mysterious Python pyc file corruption problems
Terry Jan Reedy
tjreedy at udel.edu
Thu May 16 22:52:19 CEST 2013
On 5/16/2013 2:04 PM, Barry Warsaw wrote:
> No, it's all different kinds of machines, at different times, on different
> files. So far, there's no rhyme or reason to the corruptions that I can
If the corruption only happens on Ubuntu, that would constitute 'rhyme'
;-). I realize that asking for reports on other systems is part of the
reason you posted, but I don't remember seeing any others yet.
> We're trying to instrument things to collect more data when these
> failures do occur.
Do failures only occur during compileall process? (or whatever
substitute you use).
At the end of py_compile.complile, after the with block that opens,
writes, flushes, and closes, you could add
with open(cfile, 'rb') as fc: <read header and unmarshal rest>
This would be a high-level write and verify.
Verify would be a bit faster if marshal.dump were replaced by
marshal.dumps + write to keep alive the string version of the code
object. Then the codeobject comparison in the verify step would be
replaced by string comparison.
You could also read and verify (by unmarshal) after the compile-all
process (faster than importing).
More information about the Python-Dev