zc.buildout, recipe downloading dev eggs
Howdy! I apologize if this isn't the place to ask about buildout, but it is listed as the support address in PyPI. I use zc.buildout, as many Zope / Plone and other Python developers do nowadays to help in deployment of our code. We are abandoning SVN as version control, and for now are using BZR with a shell script to pull externals from SVN. As BZR is a Python DSCM, we want to use it api-wise in a buildout recipe, as an alternative to svn:externals, as BZR's externals-like feature is a bit slow in coming, and also a recipe could support a variety of SCM by URI. My concern is that, of course, buildout expects dev-eggs to be in src/ when its' run, though dev products and other bits can come as dependencies of eggs and via recipes. What I want is to be able to download dev eggs in a recipe, which is applied before the eggs are sought. I'm glad to help, but expect that approaching folks with setuptools and esp buildout expertise, via e-mail, as a group, might be a good start. I'd be glad to help document how to do this and what situations it benefits. Best, and TIA! J -- Robert Half - "When one teaches, two learn."
On Oct 25, 2008, at 1:08 AM, Justin Ryan wrote: ...
I apologize if this isn't the place to ask about buildout, but it is listed as the support address in PyPI.
This is the right place. ...
My concern is that, of course, buildout expects dev-eggs to be in src/ when its' run,
No it doesn't. buildout is based on setuptools, which is based on distutils and builds on any mechanisms they provide, which are very flexible. Use of src directories is a common and generally convenient convention.
though dev products and other bits can come as dependencies of eggs and via recipes.
What I want is to be able to download dev eggs in a recipe, which is applied before the eggs are sought.
I'm not sure what you're saying. You can certainly create develop eggs in a recipe. Buildout "prefers" develop eggs over non-develop eggs, so it will use a develop egg rather than a regular egg if both satisfy given requirements, even develop eggs with lower version numbers. Note that this policy sometimes gets buildout into trouble since system installs often have the same type as develop eggs in setuptools. I'm considering having buildout only prefer develop eggs found in it's develop-eggs directory. I'm also, belatedly, planning to add an option to buildout to ignore site packages, which would also avoid this problem. Jim -- Jim Fulton Zope Corporation
On Sat, Oct 25, 2008 at 8:20 AM, Jim Fulton
On Oct 25, 2008, at 1:08 AM, Justin Ryan wrote: ...
I apologize if this isn't the place to ask about buildout, but it is listed as the support address in PyPI.
This is the right place.
Awesome. :)
...
My concern is that, of course, buildout expects dev-eggs to be in src/
when its' run,
No it doesn't. buildout is based on setuptools, which is based on distutils and builds on any mechanisms they provide, which are very flexible. Use of src directories is a common and generally convenient convention.
A great point. I meant to say, it expects them to be accessible before any parts are built out, AFAIK.
though dev products and other bits can come as dependencies of eggs and
via recipes.
What I want is to be able to download dev eggs in a recipe, which is applied before the eggs are sought.
I'm not sure what you're saying.
You can certainly create develop eggs in a recipe. Buildout "prefers" develop eggs over non-develop eggs, so it will use a develop egg rather than a regular egg if both satisfy given requirements, even develop eggs with lower version numbers.
It seems that I would have to run buildout twice for the develop eggs to be available, if they are downloaded in a recipe, but perhaps I'm working on an incorrect assumption. After sending this message, I started wondering if a plugin would be more appropriate than a recipe, using a recipe just seems simpler at the outset because there are existing recipes to start with, at least for SVN.
Note that this policy sometimes gets buildout into trouble since system installs often have the same type as develop eggs in setuptools. I'm considering having buildout only prefer develop eggs found in it's develop-eggs directory. I'm also, belatedly, planning to add an option to buildout to ignore site packages, which would also avoid this problem.
Seems to me that virtualenv is the answer to this problem, but it's not perfect. -- Robert Half - "When one teaches, two learn."
On Oct 25, 2008, at 1:55 PM, Justin Ryan wrote:
My concern is that, of course, buildout expects dev-eggs to be in src/ when its' run,
No it doesn't. buildout is based on setuptools, which is based on distutils and builds on any mechanisms they provide, which are very flexible. Use of src directories is a common and generally convenient convention.
A great point. I meant to say, it expects them to be accessible before any parts are built out, AFAIK.
No. buildout doesn't care when develop eggs are created. If you use the buildout develop option to create develop eggs, then they are created early. If you use the zc.recipe.egg:develop recipe, or some other recipe that creates develop eggs, then you can create them later.
It seems that I would have to run buildout twice for the develop eggs to be available, if they are downloaded in a recipe, but perhaps I'm working on an incorrect assumption.
You are. See above.
After sending this message, I started wondering if a plugin would be more appropriate than a recipe, using a recipe just seems simpler at the outset because there are existing recipes to start with, at least for SVN.
Recipes work find for your use case afaict.
Note that this policy sometimes gets buildout into trouble since system installs often have the same type as develop eggs in setuptools. I'm considering having buildout only prefer develop eggs found in it's develop-eggs directory. I'm also, belatedly, planning to add an option to buildout to ignore site packages, which would also avoid this problem.
Seems to me that virtualenv is the answer to this problem, but it's not perfect.
It would be wildly simpler to just get buildout to ignore site- packages. :) Jim -- Jim Fulton Zope Corporation
On Sat, Oct 25, 2008 at 11:05 AM, Jim Fulton
On Oct 25, 2008, at 1:55 PM, Justin Ryan wrote:
My concern is that, of course, buildout expects dev-eggs to be in src/ when its' run,
No it doesn't. buildout is based on setuptools, which is based on distutils and builds on any mechanisms they provide, which are very flexible. Use of src directories is a common and generally convenient convention.
A great point. I meant to say, it expects them to be accessible before any parts are built out, AFAIK.
No. buildout doesn't care when develop eggs are created. If you use the buildout develop option to create develop eggs, then they are created early. If you use the zc.recipe.egg:develop recipe, or some other recipe that creates develop eggs, then you can create them later.
Ah, fantastic, I came to an understanding of this last week or so, but some example usage I found led me to believe it wouldn't work. Perfect! Thanks.
It seems that I would have to run buildout twice for the develop eggs to
be available, if they are downloaded in a recipe, but perhaps I'm working on an incorrect assumption.
You are. See above.
After sending this message, I started wondering if a plugin would be more
appropriate than a recipe, using a recipe just seems simpler at the outset because there are existing recipes to start with, at least for SVN.
Recipes work find for your use case afaict.
w00t! :)
Note that this policy sometimes gets buildout into trouble since system
installs often have the same type as develop eggs in setuptools. I'm considering having buildout only prefer develop eggs found in it's develop-eggs directory. I'm also, belatedly, planning to add an option to buildout to ignore site packages, which would also avoid this problem.
Seems to me that virtualenv is the answer to this problem, but it's not perfect.
It would be wildly simpler to just get buildout to ignore site-packages. :)
If you say so, I believe it. Looking forward to new func! :) Thanks again! J -- Robert Half - "When one teaches, two learn."
On Sat, Oct 25, 2008 at 11:05 AM, Jim Fulton
wrote:
No. buildout doesn't care when develop eggs are created. If you use the buildout develop option to create develop eggs, then they are created early. If you use the zc.recipe.egg:develop recipe, or some other recipe that creates develop eggs, then you can create them later.
Ah, fantastic, I came to an understanding of this last week or so, but some example usage I found led me to believe it wouldn't work.
Perfect! Thanks.
OK, I gave this a shot in a simple case, a buildout you can find here: http://dev.vongogo.net/bzr/repos/buildouts/getpaid.buildout/ The develop option in getpaid.cfg seems to be ignored entirely, I don't see any messages like "We have a develop egg", and it fails to find a distribution for yoma.batching, which is in the develop list, though it finds others which are available in PyPI. If I'm working on another false assumption, or parsed something from you or the documentation wrong, please correct. I just want to be able to deploy in this style. :) Best, J -- Robert Half - "When one teaches, two learn."
On closer inspection, the zc.recipe.egg documentation doesn't make any
mention of this feature, though I think it's a great idea and hats off
to the author if it was originally intended to work that way but
doesn't, or maybe broke at some point because a contributor didn't
understand the feature.
If there's some work to be done, I'm happy to help.
I really want this style of buildout usage to be possible.
Let's work together! Our Foundation's goals line up with Zope extremeley well!
:)
Peace,
J
On Sun, Oct 26, 2008 at 12:04 PM, Justin Ryan
On Sat, Oct 25, 2008 at 11:05 AM, Jim Fulton
wrote: No. buildout doesn't care when develop eggs are created. If you use the buildout develop option to create develop eggs, then they are created early. If you use the zc.recipe.egg:develop recipe, or some other recipe that creates develop eggs, then you can create them later.
Ah, fantastic, I came to an understanding of this last week or so, but some example usage I found led me to believe it wouldn't work.
Perfect! Thanks.
OK, I gave this a shot in a simple case, a buildout you can find here:
http://dev.vongogo.net/bzr/repos/buildouts/getpaid.buildout/
The develop option in getpaid.cfg seems to be ignored entirely, I don't see any messages like "We have a develop egg", and it fails to find a distribution for yoma.batching, which is in the develop list, though it finds others which are available in PyPI.
If I'm working on another false assumption, or parsed something from you or the documentation wrong, please correct. I just want to be able to deploy in this style. :)
Best,
J
-- Robert Half - "When one teaches, two learn."
-- Robert Half - "When one teaches, two learn."
Jim? Anyone? I agree the behaviour we discussed makes sense, but I'm
unclear why it doesn't work as discussed, it isn't documented, and I
haven't found a clear indicator in the source that it should work,
though I clearly could learn a bit more about buildout recipes.
On Wed, Oct 29, 2008 at 8:34 PM, Justin Ryan
On closer inspection, the zc.recipe.egg documentation doesn't make any mention of this feature, though I think it's a great idea and hats off to the author if it was originally intended to work that way but doesn't, or maybe broke at some point because a contributor didn't understand the feature.
If there's some work to be done, I'm happy to help.
I really want this style of buildout usage to be possible.
Let's work together! Our Foundation's goals line up with Zope extremeley well!
:)
Peace,
J
On Sun, Oct 26, 2008 at 12:04 PM, Justin Ryan
wrote: On Sat, Oct 25, 2008 at 11:05 AM, Jim Fulton
wrote: No. buildout doesn't care when develop eggs are created. If you use the buildout develop option to create develop eggs, then they are created early. If you use the zc.recipe.egg:develop recipe, or some other recipe that creates develop eggs, then you can create them later.
Ah, fantastic, I came to an understanding of this last week or so, but some example usage I found led me to believe it wouldn't work.
Perfect! Thanks.
OK, I gave this a shot in a simple case, a buildout you can find here:
http://dev.vongogo.net/bzr/repos/buildouts/getpaid.buildout/
The develop option in getpaid.cfg seems to be ignored entirely, I don't see any messages like "We have a develop egg", and it fails to find a distribution for yoma.batching, which is in the develop list, though it finds others which are available in PyPI.
If I'm working on another false assumption, or parsed something from you or the documentation wrong, please correct. I just want to be able to deploy in this style. :)
Best,
J
-- Robert Half - "When one teaches, two learn."
-- Robert Half - "When one teaches, two learn."
-- Robert Half - "When one teaches, two learn."
participants (2)
-
Jim Fulton
-
Justin Ryan