[Python-Dev] how important is setting co_filename for a module being imported to what __file__ is set to?

[Gregory P. Smith]

On Sun, Aug 30, 2009 at 5:24 PM, Guido van Rossum <guido at python.org> wrote:

> On Sun, Aug 30, 2009 at 4:28 PM, Brett Cannon<brett at python.org> wrote:
> > I am going through and running the entire test suite using importlib
> > to ferret out incompatibilities. I have found a bunch, although all
> > rather minor (raising a different exception typically; not even sure
> > they are worth backporting as anyone reliant on the old exceptions
> > might get a nasty surprise in the next micro release), and now I am
> > down to my last failing test suite: test_import.
> >
> > Ignoring the execution bit problem (http://bugs.python.org/issue6526
> > but I have no clue why this is happening), I am bumping up against
> > TestPycRewriting.test_incorrect_code_name. Turns out that import
> > resets co_filename on a code object to __file__ before exec'ing it to
> > create a module's namespace in order to ignore the file name passed
> > into compile() for the filename argument. Now I can't change
> > co_filename from Python as it's a read-only attribute and thus can't
> > match this functionality in importlib w/o creating some custom code to
> > allow me to specify the co_filename somewhere (marshal.loads() or some
> > new function).
> >
> > My question is how important is this functionality? Do I really need
> > to go through and add an argument to marshal.loads or some new
> > function just to set co_filename to something that someone explicitly
> > set in a .pyc file? Or I can let this go and have this be the one
> > place where builtins.__import__ and importlib.__import__ differ and
> > just not worry about it?
>
> ISTR that Bill Janssen once mentioned a file replication mechanism
> whereby there were two names for each file: the "canonical" name on a
> replicated read-only filesystem, and the longer "writable" name on a
> unique master copy. He ended up with the filenames in the .pyc files
> being pretty bogus (since not everyone had access to the writable
> filesystem). So setting co_filename to match __file__ (i.e. the name
> under which the module is being imported) would be a nice service in
> this case.
>
> In general this would happen whenever you pre-compile a bunch of .py
> files to .pyc/.pyo and then copy the lot to a different location. Not
> a completely unlikely scenario.


8-9 years ago while using py2exe on windows to create stand along binaries
out of Python programs for distribution we ran into this issue... The
compiled .pyc's that py2exe bundles up contained the pathname to the source
code on the development build system.  When you get a stacktrace python
would look for the source code based on those.... Really horrible if your
build system used a windows drive letter other than C such as D: as it could
cause windows to pop up a dialog asking the user to insert a CD or spin up a
spun down optical disc or ask for a floppy, etc. ;)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090831/7907d6ef/attachment.htm>