[Distutils] buildout: several fold performance increases

[Reinout van Rees]

On 22-01-12 21:42, Ross Patterson wrote:
> Reinout van Rees<reinout at vanrees.org>  writes:
>
>> >  On 22-01-12 02:37, Ross Patterson wrote:
>>> >>  Also, with this approach, it may be possible to do
>>> >>  global dist caching which can be updated as things are changed by
>>> >>  hooking into zc.buildout.easy_install's develop(), install(), and
>>> >>  build().  Thoughts?
>> >
>> >  Would any work you do in this area make it easier to handle selected
>> >  global packages? The hard point there seems to me to keep track of
>> >  what is coming from where. When buildout has a firmer grip itself on
>> >  where what dist is, doing advanced tricks with them might be easier.
> Not quite following you yet.  Can you elaborate?  If you could also
> provide examples, that would be great.

Assumption: if you look at a global dist cache, that means buildout 
itself keeps really good track of where every dist is. It doesn't have 
to rely on setuptools internals, for instance. Buildout knows where 
every dist it knows about is.

Now, what I saw when trying out buildout 1.5 and telling it to use a 
couple of globally installed python packages, was that it would put the 
system python's library dir quite near the top of buildout's sys.path if 
that was where it found one of the packages that I asked it about.

Apparently it is hard to place such a dir at the end of sys.path for the 
current 1.5 buildouts.

=> My thought "if buildout really keeps track itself where everything 
is, perhaps it can build a better sys.path list".


(Perhaps it is still unclear: in that case please disregard. It was just 
a brainstormy thought.)


Reinout

-- 
Reinout van Rees                    http://reinout.vanrees.org/
reinout at vanrees.org             http://www.nelen-schuurmans.nl/
"If you're not sure what to do, make something. -- Paul Graham"