[Distutils] Experience of setuptools' cache design

[Phillip J. Eby]

At 01:31 PM 1/18/2008 -0500, Jim Fulton wrote:

>On Jan 18, 2008, at 1:04 PM, Phillip J. Eby wrote:
>
>>At 12:50 PM 1/18/2008 -0500, Jim Fulton wrote:
>>
>>>On Jan 18, 2008, at 12:32 PM, Phillip J. Eby wrote:
>>>
>>>>At 10:13 AM 1/18/2008 -0500, Jim Fulton wrote:
>>>...
>>>
>>>Can you briefly explain or provide a link to something that explains
>>>the performance improvement?
>>
>>Fewer directories on sys.path = better import performance, compared
>>to individually putting a series of .egg directories on sys.path.
>
>Ah, I didn't understand what you meant by "unzip the contents to the
>target directory".
>
>So the idea is that you'd merge the contents of multiple zip files
>into a single "target" directory.

Correct.


>This seems rather messy to me.

Well, it's what we all used to do before eggs came along, and what 
most packaging systems still do now.  :)


>   It doesn't appear to be compatible
>with multi-version installs.

It *can* be compatible, with certain limitations.


>   Also, if a newer egg version removes a
>file, the removed file will be left installed after an upgrade.  If
>two eggs provide the same file, files will be overridden. Admittedly,
>this is a somewhat pathological case, but the overriding seems to
>compound the pathology.

Again, this is all true without eggs now.  Clearly, eggs have spoiled 
you tremendously.  ;-)

Seriously, though, if buildout is tracking what files get installed 
or overwritten during unzipping, you can manage all of this, just as 
you presumably do for any other sort of installation recipe, no?



>As described in a separate thread, I'm going to add an option to
>buildout so buildout users can explicitly define a directory to use as
>a setuptools cache.  In that case, zip-safe eggs can remain zipped
>even if they have extensions.

If you want to be really safe/careful, you can pre-extract the eggs 
to the cache, thereby avoiding any runtime permission problems.