[Distutils] using python generated by buildout to run buildout

Łukasz Nowak email at lnowak.com
Fri Oct 15 13:44:53 CEST 2010


W dniu 6 października 2010 19:20 użytkownik Jim Fulton <jim at zope.com> napisał:
> 2010/10/6 Łukasz Nowak <email at lnowak.com>:
>> Running with python /home/luke/pybuildout/parts/python2.4/bin/python2.4
> Sounds cool!
>> And so far I am happy, I am using correct python version.
>> What I'd like to develop is to make those steps automatically, so that
>> after running:
>> curl -s http://svn.zope.org/*checkout*/zc.buildout/trunk/bootstrap/bootstrap.py
>> | python -S -
>> bin/buildout
>> Such steps would happen automagically:
>>  1 normal bootstrap with system python
>>  2 compile python as needed
>>  3 install new bin/buildout
>>  4 re-run bin/buildout
>>  5 compile python as signature change
>>  6 install new bin/buildout (which is the same)
>>  7 re-run bin/buildout
>>  8 do not compile python, as signature had not changed, continue...
>> Theoretically steps 5,6 and 7 could be avoided, but lets say I am
>> purist -- I do not trust that python compiled by buildout which is
>> running python I do not trust is bad :)
>> Easy question: Is there ability for buildout to do smart rebootstrap
>> in one run, with selecting part which contains python?
> No.  It might be worthwhile to build in support for this use case.
>> If no I am ready to develop such thing.
> Cool.

So I did. All those steps are done by one extension, available on pypi:


Happy hacking :)

>> So I read a bit about
>> extensions - I can hook before and after buildout is run. Extensions
>> have access to buildout object, I think they can play with buildout
>> externals a lot, including re-running buildout (I saw in output that
>> buildout re-runs itself after upgrading himself).
>> Are extension the way to go with such thing?
> Extensions are a valuable experiment, but they aren't
> really fleshed out.
> The mechanism is very simple, which is good.  To make it more
> valuable would require providing more hook points in buildout itself.
> As you point out though, you could convievably use the unload
> extension to cause the buildout to be rerun.

For now, for simplicity, I put everything in load. I need to "feel"
zc.buildout internals more to be sure, what is needed.

>> If needed I am even ready to play a bit more with zc.buildout
>> internals code to extend if required, but according to my
>> understanding how buildout works there are enough places I can "hook"
>> to.
> You may be right.  It sounds like a good next step.
> You could also propose additional apis that might help your
> extension.

I feel it is too early for me to propose any API to zc.buildout.

For now I focused on producing something dirty, but working well enough.

Maybe in some day, after receiving feedback, I will really know what I need.

For *now* such methods might be cool:

 * isBuildoutUpgradable -- which tells, if it is possible to do
buildout upgrade, to not try to upgrade non local buildout
 * reinstallBuildout -- which reinstalls zc.buildout and
setuptools/distribute, with accepting executable parameter, but
preserving current versions
 * restartBuildout -- which restart buildout process again

But as I said before -- I'd prefer such extension to grow from infant
to kidnergarten before proposing anything.

>> Any other comments?
> Only that your use case is not that uncommon.  Perhaps there
> should be a built in option to have buildout build it's own Python.
> One thought is that building Python is expensive enough that you'd like
> to share the work accross buildouts.  *Maybe* it would be interesting
> to point buildout at a separate Python buildout.  It would look at the
> Python buildout and build *it* if necessary and then use the resulting
> Python.

My extensions supports this case. The part which will be reused to
reboostrap buildout just need executable parameter, that is all. So
then if there would be any simple recipe, and executable pointing to
another python -- it will be picked up.

> Jim
> --
> Jim Fulton

Łukasz Nowak IT Specialist email at lnowak.com http://lnowak.com/
Skype: Shufla jid: shufla at jabster.pl gg: 1157726
``Use the Source, Luke...''

More information about the Distutils-SIG mailing list