Error with co_filename when loading modules from zip file

[Bob Rossi]

On Mon, Mar 05, 2012 at 02:22:55PM -0800, Vinay Sajip wrote:
> On Mar 5, 8:36 pm, Bob <b... at brasko.net> wrote:
> 
> > The logging package gets the filename and line number
> > of the calling function by looking at two variables, the filename
> > of the frame in the stack trace and the variable logging._srcfile.
> > The comparison is done in logging/__init__.py:findCaller.
> >
> 
> The _srcfile is computed in logging/__init__.py - can you see which of
> the paths it takes when computing _srcfile?

Yes, it's this one,
elif __file__[-4:].lower() in ['.pyc', '.pyo']:
    _srcfile = __file__[:-4] + '.py'

_srcfile I beleive is correct. It's filename that isn't IMHO.

> > I've tried putting only the pyc files, only the py files
> > and both in the zip file.
> 
> I think the filename info might be stored in the .pyc from when you
> ran it outside the .zip. If you delete all .pyc files and only
> have .py in the .zip, what happens?

Nice one.

I tried putting only pyc file, only py files and both
in the zip. But I never tried putting the py files in the zip
and deleting the pyc files. That makes it work as I'm guessing
it has to recompile the python bytecode, making the filename
and _srcfile match.

The problem with this approach, is that it's less efficient
to store the pyc files, since I've got to recompile them
on startup, right?

I found this issue,
  http://bugs.python.org/issue6811
and this related patch,
  http://hg.python.org/cpython/rev/5deb2094f033
that I think might address this issue. Although that's using 3.3
which isn't released yet.

This is probably an issue that could be addressed in the logging
library.  Comparing the compiled in filename
(which is determined at compile time) and the source file name
(which is determined at module load time) doesn't seem to
play well when you are moving the interpreter around in a zip file.
I don't think one would expect it to.

One solution would be to look to see if the filename ends in
pythonNM[.zip]/logging/__init__.py

Any suggestions?

Thanks,
Bob