environment variables in buildout config files?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
I just converted Mailman 3 to use zc.buildout. Thanks to Gary Poster for providing some good examples and the impetus for my force vacation hacking. :)
I especially like being able to specify defaults in my ~/.buildout/ default.cfg file. One thing that bugs me though is that I can't use environment variables like $HOME there. This is a minor pain because I can't share my .buildout directory between Linux and OS X machines (say, via version control). It appears as though I have to be explicit in the path, with /Users/barry on OS X and /home/barry on Linux.
Similarly, in my buildout.cfg file, if I include a develop field pointing to some source directories (other than .), I can't use $HOME/ projects/foo/bar here.
Any thoughts on supporting environment variables in .cfg files? Or do they work and I'm just experiencing operator error? ;)
A couple of other minor questions. In my ~/.buildout/default.cfg file, I have
[buildout] eggs-directory=/Users/barry/.buildout/eggs download-cache=/Users/barry/.buildout/download-cache
This is great because I can share that stuff between lots of development branches. However buildout will create the eggs directory if needed, but complain that download-cache doesn't exist. It seems like buildout could/should create download-cache on demand.
I'm also wondering, does it make sense to put develop-eggs and parts in say ~/.buildout? I kind of dislike having artifacts like these live in my version controlled source directory (well, there's bin too, but I can kind of live with that). What's the best way of doing that?
Thanks, Barry
On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
I just converted Mailman 3 to use zc.buildout. Thanks to Gary Poster for providing some good examples and the impetus for my force vacation hacking. :)
I especially like being able to specify defaults in my ~/.buildout/ default.cfg file. One thing that bugs me though is that I can't use environment variables like $HOME there.
Seconded: https://bugs.launchpad.net/zc.buildout/+bug/190260
This is a minor pain because I can't share my .buildout directory between Linux and OS X machines (say, via version control). It appears as though I have to be explicit in the path, with /Users/barry on OS X and /home/barry on Linux.
Similarly, in my buildout.cfg file, if I include a develop field pointing to some source directories (other than .), I can't use $HOME/ projects/foo/bar here.
Any thoughts on supporting environment variables in .cfg files? Or do they work and I'm just experiencing operator error? ;)
AFAIK, no.
I'm also wondering, does it make sense to put develop-eggs and parts in say ~/.buildout?
No, since those are specific to a particular project and shouldn't be shared.
(Actually, I don't understand what 'develop-eggs' is for, other than that removing it tends to fix weird problems after switching from a develop-egg to a normal egg.)
I kind of dislike having artifacts like these live in my version controlled source directory (well, there's bin too, but I can kind of live with that). What's the best way of doing that?
echo 'bin parts develop-eggs' > .csvignore svn propedit svn:ignore . bzr ignore bin parts develop-eggs or the equivalent for the VCS of your choice.
HTH, Marius Gedminas
On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote:
Thanks for answering these questions. :)
(Actually, I don't understand what 'develop-eggs' is for,
It is a place where develop eggs created by the buildout are placed so they can be found when looking for eggs to satisfy dependencies.
other than that removing it tends to fix weird problems after switching from a develop-egg to a normal egg.)
I fixed a bug in managing develop egg links some time ago. If you are still having problems, please submit a bug.
Jim
-- Jim Fulton Zope Corporation
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jan 8, 2009, at 3:02 PM, Jim Fulton wrote:
Thanks for answering these questions. :)
(Actually, I don't understand what 'develop-eggs' is for,
It is a place where develop eggs created by the buildout are placed so they can be found when looking for eggs to satisfy dependencies.
Thanks for the answer!
Barry
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote:
On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
I just converted Mailman 3 to use zc.buildout. Thanks to Gary Poster for providing some good examples and the impetus for my force vacation hacking. :)
I especially like being able to specify defaults in my ~/.buildout/ default.cfg file. One thing that bugs me though is that I can't use environment variables like $HOME there.
Seconded: https://bugs.launchpad.net/zc.buildout/+bug/190260
Thanks. I've just added a comment on that bug, and have subscribed to it.
This is a minor pain because I can't share my .buildout directory between Linux and OS X machines (say, via version control). It appears as though I have to be explicit in the path, with /Users/barry on OS X and /home/barry on Linux.
Similarly, in my buildout.cfg file, if I include a develop field pointing to some source directories (other than .), I can't use $HOME/ projects/foo/bar here.
Any thoughts on supporting environment variables in .cfg files? Or do they work and I'm just experiencing operator error? ;)
AFAIK, no.
I'm also wondering, does it make sense to put develop-eggs and parts in say ~/.buildout?
No, since those are specific to a particular project and shouldn't be shared.
Ok.
(Actually, I don't understand what 'develop-eggs' is for, other than that removing it tends to fix weird problems after switching from a develop-egg to a normal egg.)
I kind of dislike having artifacts like these live in my version controlled source directory (well, there's bin too, but I can kind of live with that). What's the best way of doing that?
echo 'bin parts develop-eggs' > .csvignore svn propedit svn:ignore . bzr ignore bin parts develop-eggs or the equivalent for the VCS of your choice.
Actually, it's not so much 'bzr stat' that I care about since that's easily ignored. It's 'ls' that I care about since that's tougher to ignore. I guess I could use dot-prefixes. ;)
Thanks Barry
On Thu, Jan 08, 2009 at 04:12:11PM -0500, Barry Warsaw wrote:
On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote:
On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
I'm also wondering, does it make sense to put develop-eggs and parts in say ~/.buildout?
No, since those are specific to a particular project and shouldn't be shared.
Ok.
(Actually, I don't understand what 'develop-eggs' is for, other than that removing it tends to fix weird problems after switching from a develop-egg to a normal egg.)
After Jim's explanation (thanks!) I don't see a reason why develop-eggs couldn't also be shared... no, wait, perhaps I see one.
Imagine project A using a develop-egg of a *branch* of zope.frobozz, while project B uses a develop-egg of the *trunk* of zope.frobozz. In that case /path/to/project-A/develop-eggs/zope.frobozz.egg-link will contain a link to ~/src/zope.frobozz/random-feature-branch, while /path/to/project-B/develop-eggs/zope.frobozz.egg-link would containt a link to ~/src/zope.frobozz/trunk. Same filename, different content -- ergo, develop-eggs cannot be shared between projects.
I have a couple of source trees in my home directory that contain an actual example of this.
I kind of dislike having artifacts like these live in my version controlled source directory (well, there's bin too, but I can kind of live with that). What's the best way of doing that?
echo 'bin parts develop-eggs' > .csvignore svn propedit svn:ignore . bzr ignore bin parts develop-eggs or the equivalent for the VCS of your choice.
Actually, it's not so much 'bzr stat' that I care about since that's easily ignored.
(in which case your wording misled me ;)
It's 'ls' that I care about since that's tougher to ignore. I guess I could use dot-prefixes. ;)
Personally, I'm more bothered by the '*.egg-info' directories created by setuptools. Also, but to a somewhat lesser extent, the 'build' and 'dist' ones created by distutils.
'bin' and 'parts' are something I need to deal with rather often. I've grown to like bin/$randomscript, and while 'parts/$whatever/$somefile' is a bit too deep and nested, I appreciate the namespacing and explicitness ("this is a directory managed by buildout; it may be wiped out at any moment; don't touch it"). 'develop-eggs' is something that I'm not personally interested in and wouldn't mind hiding with a dot-prefix, but since I can't do that with 'build' and 'dist', I don't see much point. As long as it doesn't interfere with my tab-completion, I'm fine.
Regards, Marius Gedminas
On Jan 8, 2009, at 4:28 PM, Marius Gedminas wrote:
On Thu, Jan 08, 2009 at 04:12:11PM -0500, Barry Warsaw wrote:
On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote:
On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
I'm also wondering, does it make sense to put develop-eggs and parts in say ~/.buildout?
No, since those are specific to a particular project and shouldn't be shared.
Ok.
(Actually, I don't understand what 'develop-eggs' is for, other than that removing it tends to fix weird problems after switching from a develop-egg to a normal egg.)
After Jim's explanation (thanks!) I don't see a reason why develop- eggs couldn't also be shared... no, wait, perhaps I see one.
Imagine project A using a develop-egg of a *branch* of zope.frobozz, while project B uses a develop-egg of the *trunk* of zope.frobozz. In that case /path/to/project-A/develop-eggs/zope.frobozz.egg-link will contain a link to ~/src/zope.frobozz/random-feature-branch, while /path/to/project-B/develop-eggs/zope.frobozz.egg-link would containt a link to ~/src/zope.frobozz/trunk. Same filename, different content -- ergo, develop-eggs cannot be shared between projects.
I have a couple of source trees in my home directory that contain an actual example of this.
Right. That's the main reason they shouldn't be shared.
My explanation was also incomplete. If you use the zc.recipe.egg:custom recipe to do a custom build of a binary, then the resulting binary egg is also put in develop-eggs because the egg depends on the buildout configuration.
Jim
-- Jim Fulton Zope Corporation
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jan 8, 2009, at 4:28 PM, Marius Gedminas wrote:
After Jim's explanation (thanks!) I don't see a reason why develop- eggs couldn't also be shared... no, wait, perhaps I see one.
Imagine project A using a develop-egg of a *branch* of zope.frobozz, while project B uses a develop-egg of the *trunk* of zope.frobozz. In that case /path/to/project-A/develop-eggs/zope.frobozz.egg-link will contain a link to ~/src/zope.frobozz/random-feature-branch, while /path/to/project-B/develop-eggs/zope.frobozz.egg-link would containt a link to ~/src/zope.frobozz/trunk. Same filename, different content -- ergo, develop-eggs cannot be shared between projects.
I have a couple of source trees in my home directory that contain an actual example of this.
That makes sense. I don't do this, but it definitely makes sense.
I kind of dislike having artifacts like these live in my version controlled source directory (well, there's bin too, but I can kind of live with that). What's the best way of doing that?
echo 'bin parts develop-eggs' > .csvignore svn propedit svn:ignore . bzr ignore bin parts develop-eggs or the equivalent for the VCS of your choice.
Actually, it's not so much 'bzr stat' that I care about since that's easily ignored.
(in which case your wording misled me ;)
Sorry ;)
It's 'ls' that I care about since that's tougher to ignore. I guess I could use dot-prefixes. ;)
Personally, I'm more bothered by the '*.egg-info' directories created by setuptools. Also, but to a somewhat lesser extent, the 'build' and 'dist' ones created by distutils.
Yeah, I hate all of those too!
'bin' and 'parts' are something I need to deal with rather often. I've grown to like bin/$randomscript, and while 'parts/$whatever/$somefile' is a bit too deep and nested, I appreciate the namespacing and explicitness ("this is a directory managed by buildout; it may be wiped out at any moment; don't touch it"). 'develop-eggs' is something that I'm not personally interested in and wouldn't mind hiding with a dot-prefix, but since I can't do that with 'build' and 'dist', I don't see much point. As long as it doesn't interfere with my tab- completion, I'm fine.
bin is a tough one. I love its convenience. parts I've never used.
setup_requires dependencies also show up here (I always see the bzr egg because I setup-depend on setuptools_bzr).
One other thing that bugs me about these is that it's not obvious from an ls what you can actually delete because it's an artifact of setuptools/buildout/etc and what is part of your package. I mean, you have to know that it's safe to delete bin, dist, and build right?
setup.py has a 'clean' command but it's not enough. Maybe if buildout added a --clean option or something that would be better. What I really want is a simple way to revert a tree back to its pristine version-controlled layout, without all the build, develop, test artifacts.
On Thu, Jan 08, 2009 at 04:37:05PM -0500, Barry Warsaw wrote:
One other thing that bugs me about these is that it's not obvious from an ls what you can actually delete because it's an artifact of setuptools/buildout/etc and what is part of your package. I mean, you have to know that it's safe to delete bin, dist, and build right?
Or you could see which files are ignored by your VCS. E.g. with bzr you'd do 'rm `bzr unknowns`', and I'm sure you could write a one-liner to work with 'svn st -v'.
Or you could get a fresh checkout, or use svn export.
setup.py has a 'clean' command but it's not enough. Maybe if buildout added a --clean option or something that would be better. What I really want is a simple way to revert a tree back to its pristine version-controlled layout, without all the build, develop, test artifacts.
I'd say this is up to your version-control system to implement this, then. It doesn't feel right to put this into buildout. E.g. how would it know that it's OK to erase ./eggs, but not OK to erase ~/.buildout/my-shared-egg-cache/, if the difference is just an option in ~/.buildout/default.cfg? Sure, you could see which files are inside the tree and which ones are outside, but it rapidly descends into guessing. Which you should refuse the temptation to, according to Zen.
Also, there are files littering your source tree that aren't tracked in a VCS, and weren't created by buildout (e.g. *.pyc). I don't think bin/buildout clean should remove files bin/buildout didn't create.
Marius Gedminas
participants (3)
-
Barry Warsaw -
Jim Fulton -
Marius Gedminas