[Distutils] [issue33] zipped eggs cause various problems -- perhaps set default setting to unzipped?

[Zooko O'Whielacronx]

New submission from Zooko O'Whielacronx <zooko at zooko.com>:

per http://mail.python.org/pipermail/python-dev/2008-April/078505.html

installing an egg (with easy_install or ./setup.py install) in zipped form
results in stack traces without source code lines attached. I hereby propose
that zip_safe=False should be the default, unless the package producer and/or
the package consumer specifically indicates that they prefer to zip the egg.

I might be persuaded to change my mind about this if I saw some performance
measurements which showed that zip_safe=False had a high cost. I'm skeptical
that it has a high cost, since I've used it extensively for months (habitually
specifying zip_safe=False in my setup.py files) and I haven't noticed any
performance problem. 

PJE wrote, on the python-dev thread:

"Are you using Python 2.5?  As of 2.5, the linecache module should correctly
read the source line from the present location of the source file the module was
loaded from, regardless of the file name specified in the traceback."

Yes, this was with Python 2.5.1. Note that the module was loaded from a .pyc
file inside a zipped .egg (in the failing case), or from a .pyc file in an
unzipped .egg (in the succeeding case). In the succeeding case, the name of the
.py file, which is encoded inside the .pyc file, had been rewritten during the
install process. 

 Changed 4 months ago by zooko  ¶

Here is another case that I encountered yesterday in which eggs being zipped
caused a problem: we're using py2exe to build a Windows executable. (Also we are
using py2app to build a Mac app.)

py2exe can't find Python modules inside zipped eggs, although it can of course
find them in unzipped eggs if the egg is added to the PYTHONPATH. At one point
my programming partner Rob Kinninmont was using a script named ["hatch-eggs.py"
http://allmydata.org/trac/tahoe/browser/misc/hatch-eggs.py?rev=1879]. That
script no longer appears to be needed in our build system, but at this moment I
can't figure out how we give py2exe access to the modules from the eggs that we
installed... It is possible that we've customized the build machine by setting a
distutils config file to say "always_unzip=True". I hope not, because that would
mean our build will fail in mysterious ways when run on a different machine. 

 Changed 4 months ago by zooko  ¶

Oh, there it is: our [setup.cfg
http://allmydata.org/trac/tahoe/browser/setup.cfg?rev=1938]. This means our
Windows build will work on other machines, unless they have one of our
dependencies already installed and that dependency is installed in zipped form.
In that case, we'll get some strange error quite late in the process -- possibly
something like the module not being found at run-time.

So, this is another example of how unzipped eggs Just Work in more contexts
(py2exe) than zipped ones do. 

 Changed 4 months ago by nbecker  ¶

zipped eggs also breaks pydoc:

http://permalink.gmane.org/gmane.comp.python.devel/93788

Also, not very convenient if you need to view the source or debug. 



Changed 1 month ago by zooko ¶

Here's another one:

http://mail.python.org/pipermail/distutils-sig/2008-July/009694.html
Changed 1 month ago by zooko ¶

Here's another person who had trouble which he wouldn't have had if eggs were
unzipped by default:

http://mail.python.org/pipermail/distutils-sig/2007-December/008554.html 


Okay, here's the latest entry: my co-worker Greg Hazel, not knowing about the
setup.cfg hack that I described earlier in this ticket, installed one of Tahoe's
dependencies onto a build machine, using the default behavior of easy_install. 
Therefore, later there was a mysterious run-time error in the resulting package
of Tahoe.  Unfortunately I didn't catch this and explain it to Greg right away,
so instead he spent some of his time figuring it out himself.  After
working-around the problem somehow, he reported to his manager: "fixed. you can
say thank you to python eggs for screwing up py2exe's automatic dependency
detection".

(This ticket was imported from my interim setuptools bug tracker:
http://allmydata.org/trac/setuptools/ticket/4
)

----------
messages: 85
nosy: zooko
priority: bug
status: unread
title: zipped eggs cause various problems -- perhaps set default setting to unzipped?

_______________________________________________
Setuptools tracker <setuptools at bugs.python.org>
<http://bugs.python.org/setuptools/issue33>
_______________________________________________