[Python-ideas] disabling .pyc and .pyo files
kevin.watters at gmail.com
Tue Dec 15 23:37:09 CET 2009
For what it's worth, I've got an entirely different use case than the
ones I've seen in this thread so far.
I'd like Python to read .pyo files, but not search for .py or .pyc
files. This is because we ship a py2exe app in it's "exploded" form,
where there is an .exe and a lib/ folder full of .pyos. Purely as an
optimization, it'd be nice to not have Python stat for .py and then .pyc
for every new import.
I remember glancing at Python/import.c and thinking that this could
easily be accomplished by allowing the user to customize
_PyImport_StandardFiletab at runtime--in fact there is already an
PyImport_AppendInittab; it's just not exposed to Python. With a function
like imp.set_inittab, I could get what I want with something like
imp.set_inittab(['.pyo', 'rb', imp.PY_COMPILED])
And then of course to read just .py files, you could do
imp.set_inittab([".py", "U", PY_SOURCE])
Kristján Valur Jónsson wrote:
> Hello there.
> We have a large project involving multiple perforce branches of hundreds
> of .py files each.
> Although we employ our own import mechanism for the bulk of these files,
> we do use the regular import mechanism for an essential core of them.
> Repeatedly we run into trouble because of stray .pyo (and/or .pyc)
> files. This can happen for a variety of reasons, but most often it
> occurs when .py files are being removed, or moved in the hierarchy. The
> problem is that the application will happily load and import an orphaned
> .pyo file, even though the .py file has gone or moved.
> I looked at the import code and I found that it is trivial to block the
> reading and writing of .pyo files. I am about to implement that patch
> for our purposes, thus forcing recompilation of the .py files on each
> run if so specified. This will ensure that the application will
> execute only the code represented by the checked-out .py files. But it
> occurred to me that this functionality might be of interest to other
> people than just us. I can imagine, for example, that buildbots running
> the python regression testsuite might be running into problems with
> stray .pyo files from time to time.
> Do you think that such a command line option would be useful for Python
> at large?
> Python-ideas mailing list
> Python-ideas at python.org
More information about the Python-ideas