[issue33] zipped eggs cause various problems -- perhaps set default setting to unzipped?
New submission from Zooko O'Whielacronx <zooko@zooko.com>: per http://mail.python.org/pipermail/python-dev/2008-April/078505.html installing an egg (with easy_install or ./setup.py install) in zipped form results in stack traces without source code lines attached. I hereby propose that zip_safe=False should be the default, unless the package producer and/or the package consumer specifically indicates that they prefer to zip the egg. I might be persuaded to change my mind about this if I saw some performance measurements which showed that zip_safe=False had a high cost. I'm skeptical that it has a high cost, since I've used it extensively for months (habitually specifying zip_safe=False in my setup.py files) and I haven't noticed any performance problem. PJE wrote, on the python-dev thread: "Are you using Python 2.5? As of 2.5, the linecache module should correctly read the source line from the present location of the source file the module was loaded from, regardless of the file name specified in the traceback." Yes, this was with Python 2.5.1. Note that the module was loaded from a .pyc file inside a zipped .egg (in the failing case), or from a .pyc file in an unzipped .egg (in the succeeding case). In the succeeding case, the name of the .py file, which is encoded inside the .pyc file, had been rewritten during the install process. Changed 4 months ago by zooko ¶ Here is another case that I encountered yesterday in which eggs being zipped caused a problem: we're using py2exe to build a Windows executable. (Also we are using py2app to build a Mac app.) py2exe can't find Python modules inside zipped eggs, although it can of course find them in unzipped eggs if the egg is added to the PYTHONPATH. At one point my programming partner Rob Kinninmont was using a script named ["hatch-eggs.py" http://allmydata.org/trac/tahoe/browser/misc/hatch-eggs.py?rev=1879]. That script no longer appears to be needed in our build system, but at this moment I can't figure out how we give py2exe access to the modules from the eggs that we installed... It is possible that we've customized the build machine by setting a distutils config file to say "always_unzip=True". I hope not, because that would mean our build will fail in mysterious ways when run on a different machine. Changed 4 months ago by zooko ¶ Oh, there it is: our [setup.cfg http://allmydata.org/trac/tahoe/browser/setup.cfg?rev=1938]. This means our Windows build will work on other machines, unless they have one of our dependencies already installed and that dependency is installed in zipped form. In that case, we'll get some strange error quite late in the process -- possibly something like the module not being found at run-time. So, this is another example of how unzipped eggs Just Work in more contexts (py2exe) than zipped ones do. Changed 4 months ago by nbecker ¶ zipped eggs also breaks pydoc: http://permalink.gmane.org/gmane.comp.python.devel/93788 Also, not very convenient if you need to view the source or debug. Changed 1 month ago by zooko ¶ Here's another one: http://mail.python.org/pipermail/distutils-sig/2008-July/009694.html Changed 1 month ago by zooko ¶ Here's another person who had trouble which he wouldn't have had if eggs were unzipped by default: http://mail.python.org/pipermail/distutils-sig/2007-December/008554.html Okay, here's the latest entry: my co-worker Greg Hazel, not knowing about the setup.cfg hack that I described earlier in this ticket, installed one of Tahoe's dependencies onto a build machine, using the default behavior of easy_install. Therefore, later there was a mysterious run-time error in the resulting package of Tahoe. Unfortunately I didn't catch this and explain it to Greg right away, so instead he spent some of his time figuring it out himself. After working-around the problem somehow, he reported to his manager: "fixed. you can say thank you to python eggs for screwing up py2exe's automatic dependency detection". (This ticket was imported from my interim setuptools bug tracker: http://allmydata.org/trac/setuptools/ticket/4 ) ---------- messages: 85 nosy: zooko priority: bug status: unread title: zipped eggs cause various problems -- perhaps set default setting to unzipped? _______________________________________________ Setuptools tracker <setuptools@bugs.python.org> <http://bugs.python.org/setuptools/issue33> _______________________________________________
On Aug 15, 2008, at 11:41 AM, Zooko O'Whielacronx wrote:
I might be persuaded to change my mind about this if I saw some performance measurements which showed that zip_safe=False had a high cost. I'm skeptical that it has a high cost, since I've used it extensively for months (habitually specifying zip_safe=False in my setup.py files) and I haven't noticed any performance problem.
I have found unzipped eggs to import *faster* yes *faster* than zipped eggs. Jim -- Jim Fulton Zope Corporation
participants (2)
-
Jim Fulton -
Zooko O'Whielacronx