Race condition when generating .pyc files

yogamatt1970 at gmail.com yogamatt1970 at gmail.com
Tue Oct 7 17:21:40 CEST 2008


I have a large body of Python code which runs on many different (Unix)
machines concurrently.  Part of the code lives in one place, but most
of it lives in directories which I find at runtime.  I only have one
copy of each Python source file and I think I'm hitting a race
condition where two hosts attempt to import the same module at the
same time.  My import fails on one of the machines and the following
exception is thrown:
EOFError: EOF read where object expected
My hypothesis is that there's contention between the two (or more)
hosts when the module's .pyc file is generated.

Possible solutions I see:
1) Running a cron job before my code executes which compiles all the
python source first.
2) Making separate copies of all the .py files for each host running
the code - I'd rather not do this, it seems like a big pain.
3) Inhibiting the generation of .pyc files altogether if that's even
possible - I saw a PEP for a possible -R flag (http://www.python.org/
dev/peps/pep-0304/) but I don't think it has been added to a released
version of Python yet plus I am stuck with Python 2.4 for the time
being.

Any and all help appreciated.
Thanks.



More information about the Python-list mailing list