[Python-ideas] Importing orphaned bytecode files (was: disabling .pyc and .pyo files)

Brett Cannon brett at python.org
Wed Dec 9 23:43:05 CET 2009

On Wed, Dec 9, 2009 at 11:27, Guido van Rossum <guido at python.org> wrote:

> On Wed, Dec 9, 2009 at 11:07 AM, Jared Grubb <jared.grubb at gmail.com>
> wrote:
> >
> > On 8 Dec 2009, at 13:44, Ben Finney wrote:
> >
> > * A new attribute ‘sys.import_orphaned_bytecode’. If set ‘True’, the
> >  interpreter follows the current behaviour. If ‘False’, any bytecode
> >  file satisfies an import only if it has a corresponding source file
> >  (where “corresponding” means “this source file would, if compiled,
> >  result in a bytecode file replacing this one”).
> >
> > One problem with a sys flag is that it's a global setting. Suppose a
> package
> > is distributed with only pyc/pyo files, then the top-level __init__.py
> might
> > flip the switch such that its sub-files can get imported from the pyc/pyo
> > files. But you wouldnt want that flag to persist beyond that.
> I'm not sure that there are any use cases that require using
> conflicting values of this setting for different packages.
Same here. This is straying into optimizations for the sake of optimizing.

>  > Another idea is to use a new file extension, which isnt the best
> solution,
> > but allows the creator to explicitly set what behavior they intended for
> > their files:
> >   * if a foo.py file exists, then use the existing foo.pyc/pyo as is done
> > today
> >   * if a foo.py file does not exist, but a foo.pyxxx exists, use it (but
> > file.pyc/pyo is never used, unlike today)
> > (pyxxx is a placeholder for whatever would be a reasonable name)
> It's a much bigger change, but using a different extension would
> probably remove the need for a flag. It would also help with some
> tools that hide .pyc/.pyo files from view (e.g. the typical
> .svnignore).

>From a Python VM perspective, the problem with this is it doesn't help
improve the situation for other VMs that have no concept of bytecode. If we
make pyc/pyo files purely an optimization for CPython (and other VMs that
choose to support the format) and not a recognized executable format on its
own (like it is now) then that would probably help prevent people from
distributing pyc/pyo files only and thus locking out the use of other VMs.

I know some people seem to think pyc/pyo fles are a good way to obfuscate
code, but it honestly isn't, IMO. But these people stand the most to lose
from us even considering changing default behavior.

In a perfect world I would make pyc/pyo files completely optional and only
an optimization that could not work w/o the corresponding source. But in a
backwards-compatible, paranoid world I would make it an opt-in flag to
ignore lone pyc/pyo files. I am +10 on the former and +1 on the latter.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20091209/f7ecd8e8/attachment.html>

More information about the Python-ideas mailing list