[Distutils] formencode as .egg in Debian ??

M.-A. Lemburg mal at egenix.com
Tue Nov 22 18:33:12 CET 2005

Phillip J. Eby wrote:
> At 01:04 AM 11/22/2005 -0600, Bob Tanner wrote:
>>On Tuesday 22 November 2005 12:15 am, Martin v. Löwis wrote:
>>>I don't think Debian should use the egg structure. It apparently relies
>>>on building a long sys.path (even though through only a single .pth
>>I'm not sure of how .eggs are implemented, but I'm going to cross-post this
>>info to the python-distutils mailing list. See below for additional comments.
>>>this adds additional costs to all import statements on startup.
>>>It gets worse if these are zipfiles, because then each import statement
>>>will have to look into each zipfile (until the import is resolved).
>>The is the opposite of what I was told  by upstream development over on
>>distutils, snippet from Phillip J. Eby <pje at telecommunity.com>:
>>Note also that in many cases, the package will be a single .egg file,
>>(analagous to a Java .jar file) rather than a directory, and files are
>>preferable to directories in most cases as they make Python import
>>processing faster.
> Yes, it's true, zipfile import processing is faster than normal import 
> processing; 

Only after *all* ZIP files on sys.path have been scanned
for their contents. The more you add to sys.path, the longer
Python takes to startup.

What's worse is that the slow-down affects the whole Python
installation - each and every application using Python will
have to scan all these ZIP files in case it tries to import
a non-existing module or one which it finds late on sys.path.

> it is in fact one of the reasons zipfile imports were added to 
> Python, because the zip directories are cached.  A zipfile import lookup is 
> a single dictionary lookup, whereas a directory import lookup requires 
> multiple stat() calls.  For all practical purposes, zipfiles added to 
> sys.path are free after the initial directory read operation.

They are "free" for long running applications only where
this caching makes sense.

> Note that the need for a .pth is a limitation caused by the requirement to 
> have packages importable at startup.  Packages installed in "multi-version" 
> or "deactivated" mode are only added to sys.path upon request and have no 
> impact on startup time.  Relatively few eggs *need* to be installed with a 
> .pth file; we are simply in a transitional period where people still expect 
> "installed" packages to be importable without an additional require() 
> operation.
> Finally, I think it's important to note that what Debian should or should 
> not use isn't really relevant to Debian's users, who will quite simply need 
> eggs for many packages.  If Debian doesn't provide them, the users will be 
> forced to obtain them elsewhere.  Over time, the number of packages that 
> users need in egg form will continue to increase, and there will be an 
> increasing number of users wanting to know why Debian can't provide 
> them.  It's perfectly reasonable not to redo existing Debian packages to 
> use eggs, but for some packages, *not* using eggs is simply not an option.

Why should "eggs" be the only way to install a package ?

Doesn't the standard "python setup.py install" work with
eggified packages anymore (meaning that the package is
installed as normal site-packages package) ?

Marc-Andre Lemburg

Professional Python Services directly from the Source  (#1, Nov 22 2005)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

More information about the Distutils-SIG mailing list