[Distutils] formencode as .egg in Debian ??

Phillip J. Eby pje at telecommunity.com
Tue Nov 22 23:42:35 CET 2005


At 10:27 PM 11/22/2005 +0100, M.-A. Lemburg wrote:
>Every single time you fire up Python and the user has not
>installed a module called "sitecustomize" (which is deliberatly
>not shipped with Python), Python will scan the complete sys.path
>for this module... and that's just one example.

Hm, I did forget about sitecustomize in this case.


>It is rather common in Python code to test for the availability
>of a faster variant by trying an import (e.g. for XML parsers)
>and then falling back to some slower emulations.

But a failed import is hardly a fast operation even with only a handful of 
directories on sys.path, so I don't see where this is hurting anything.  In 
fact, if it's "rather common", then even the zip directory read cost is 
amortized over a larger total number of operations.


> > Note by the way that "scan all these ZIP files" is a misleading term in 
> any
> > case - the files are not "scanned".  They are opened, and a small 
> amount of
> > data is read from the end of the file.  Nothing that I would consider
> > "scanning" is involved.
>
>The data read from the end of the file is the directory
>which is decoded using marshal functions. You normally
>call this scanning data.

Well, *I* normally wouldn't call it scanning, I'd call it reading, 
decoding, or marshalling.  Scanning would be *looking* for something, 
versus reading and decoding a contiguous block of data at a well-known 
location.  But oh well.


>In summary, things get slower when importing from ZIP files;
>it really only makes sense for applications that have a long
>run time and where startup is not that important, e.g.
>Zope et al.

I'm terribly curious what Python applications exist for whom:

1. Startup time is a consideration, that
2. Haven't already been refactored to a long-running process.

The only thing that occurs to me as even a possibility would be some kind 
of frequently-used system administration utility, like if you were going to 
rewrite all the bash builtin commands as Python scripts.

(And no, I'm not being sarcastic, I really *am* curious about what kind of 
tools this would be an actual issue for.)


>The main argument for using ZIP imports is to easy
>distribution of complete pure-Python packages, not
>a performance gain. You'd use one of the freeze tools
>for that, e.g. mxCGIPython which creates a single
>file Python interpreter which has a really good
>startup time due to the fact that the Python lib
>is embedded into the Python executable as static data
>and then loaded on demand by the OS as needed.

Then why was the python##.zip entry added to sys.path in Python 2.3?  My 
understanding was that it was added to allow Python to start faster by 
cutting down on extraneous stat() calls.



More information about the Distutils-SIG mailing list