environment variables in buildout config files?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I just converted Mailman 3 to use zc.buildout. Thanks to Gary Poster for providing some good examples and the impetus for my force vacation hacking. :) I especially like being able to specify defaults in my ~/.buildout/ default.cfg file. One thing that bugs me though is that I can't use environment variables like $HOME there. This is a minor pain because I can't share my .buildout directory between Linux and OS X machines (say, via version control). It appears as though I have to be explicit in the path, with /Users/barry on OS X and /home/barry on Linux. Similarly, in my buildout.cfg file, if I include a develop field pointing to some source directories (other than .), I can't use $HOME/ projects/foo/bar here. Any thoughts on supporting environment variables in .cfg files? Or do they work and I'm just experiencing operator error? ;) A couple of other minor questions. In my ~/.buildout/default.cfg file, I have [buildout] eggs-directory=/Users/barry/.buildout/eggs download-cache=/Users/barry/.buildout/download-cache This is great because I can share that stuff between lots of development branches. However buildout will create the eggs directory if needed, but complain that download-cache doesn't exist. It seems like buildout could/should create download-cache on demand. I'm also wondering, does it make sense to put develop-eggs and parts in say ~/.buildout? I kind of dislike having artifacts like these live in my version controlled source directory (well, there's bin too, but I can kind of live with that). What's the best way of doing that? Thanks, Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSWJnUHEjvBPtnXfVAQInXwP/VGi1JoXW2qMV7u1JhvVSIF2fm3WiCz01 z7M/OenoLzdpicidNY5+gfus8Y8b5qKosA/8XAhA5BM9WOmebjMuaWsR0EZiQEY5 LXwhHlEskoYJlhYRbnnCQyEfTZOIBB57kMGsgKNnRDETz4f7XWEqKvIhdBERWBhx gWtQAufcE4c= =81NH -----END PGP SIGNATURE-----
On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
I just converted Mailman 3 to use zc.buildout. Thanks to Gary Poster for providing some good examples and the impetus for my force vacation hacking. :)
I especially like being able to specify defaults in my ~/.buildout/ default.cfg file. One thing that bugs me though is that I can't use environment variables like $HOME there.
Seconded: https://bugs.launchpad.net/zc.buildout/+bug/190260
This is a minor pain because I can't share my .buildout directory between Linux and OS X machines (say, via version control). It appears as though I have to be explicit in the path, with /Users/barry on OS X and /home/barry on Linux.
Similarly, in my buildout.cfg file, if I include a develop field pointing to some source directories (other than .), I can't use $HOME/ projects/foo/bar here.
Any thoughts on supporting environment variables in .cfg files? Or do they work and I'm just experiencing operator error? ;)
AFAIK, no.
I'm also wondering, does it make sense to put develop-eggs and parts in say ~/.buildout?
No, since those are specific to a particular project and shouldn't be shared. (Actually, I don't understand what 'develop-eggs' is for, other than that removing it tends to fix weird problems after switching from a develop-egg to a normal egg.)
I kind of dislike having artifacts like these live in my version controlled source directory (well, there's bin too, but I can kind of live with that). What's the best way of doing that?
echo 'bin parts develop-eggs' > .csvignore svn propedit svn:ignore . bzr ignore bin parts develop-eggs or the equivalent for the VCS of your choice. HTH, Marius Gedminas -- If something has not yet gone wrong then it would ultimately have been beneficial for it to go wrong.
On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote: Thanks for answering these questions. :)
(Actually, I don't understand what 'develop-eggs' is for,
It is a place where develop eggs created by the buildout are placed so they can be found when looking for eggs to satisfy dependencies.
other than that removing it tends to fix weird problems after switching from a develop-egg to a normal egg.)
I fixed a bug in managing develop egg links some time ago. If you are still having problems, please submit a bug. Jim -- Jim Fulton Zope Corporation
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 8, 2009, at 3:02 PM, Jim Fulton wrote:
Thanks for answering these questions. :)
(Actually, I don't understand what 'develop-eggs' is for,
It is a place where develop eggs created by the buildout are placed so they can be found when looking for eggs to satisfy dependencies.
Thanks for the answer! Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSWZsR3EjvBPtnXfVAQK0wAP8C89AgIrrl9PZ8FM9Qq/ab+AC7eq5t1Hu MJQ9DsN6z2KgC5+4JtcNyAXdiL00UXiMvvlYyynzCeT4k4N5hMU3Qh/D5s0K6aTX GnOgYumxA+tJEL+f2dDBr0phSbd3SX6PNYoMMfb13nVfyID6XyFgdCD+dzOqHKZl b7MFIBg2370= =FeYw -----END PGP SIGNATURE-----
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote:
On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
I just converted Mailman 3 to use zc.buildout. Thanks to Gary Poster for providing some good examples and the impetus for my force vacation hacking. :)
I especially like being able to specify defaults in my ~/.buildout/ default.cfg file. One thing that bugs me though is that I can't use environment variables like $HOME there.
Seconded: https://bugs.launchpad.net/zc.buildout/+bug/190260
Thanks. I've just added a comment on that bug, and have subscribed to it.
This is a minor pain because I can't share my .buildout directory between Linux and OS X machines (say, via version control). It appears as though I have to be explicit in the path, with /Users/barry on OS X and /home/barry on Linux.
Similarly, in my buildout.cfg file, if I include a develop field pointing to some source directories (other than .), I can't use $HOME/ projects/foo/bar here.
Any thoughts on supporting environment variables in .cfg files? Or do they work and I'm just experiencing operator error? ;)
AFAIK, no.
I'm also wondering, does it make sense to put develop-eggs and parts in say ~/.buildout?
No, since those are specific to a particular project and shouldn't be shared.
Ok.
(Actually, I don't understand what 'develop-eggs' is for, other than that removing it tends to fix weird problems after switching from a develop-egg to a normal egg.)
I kind of dislike having artifacts like these live in my version controlled source directory (well, there's bin too, but I can kind of live with that). What's the best way of doing that?
echo 'bin parts develop-eggs' > .csvignore svn propedit svn:ignore . bzr ignore bin parts develop-eggs or the equivalent for the VCS of your choice.
Actually, it's not so much 'bzr stat' that I care about since that's easily ignored. It's 'ls' that I care about since that's tougher to ignore. I guess I could use dot-prefixes. ;) Thanks Barry -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSWZsK3EjvBPtnXfVAQI0tgP+IM7CeEQ2AxHBTXElbyLtwNjkuEAbK9ro PUuGYasxoyedtqG2K2Mevso2Bsjngu0agdxyjQWzqKr90g8yKEgGBfxkVM93AHlo NIb5An4MXzAlH2tJGQqv4JVlYmXwxer30n2ezT6ZVAD1S4twJMYTSIpA3q1QE2Hd f5EYrF+JahE= =KpdQ -----END PGP SIGNATURE-----
On Thu, Jan 08, 2009 at 04:12:11PM -0500, Barry Warsaw wrote:
On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote:
On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
I'm also wondering, does it make sense to put develop-eggs and parts in say ~/.buildout?
No, since those are specific to a particular project and shouldn't be shared.
Ok.
(Actually, I don't understand what 'develop-eggs' is for, other than that removing it tends to fix weird problems after switching from a develop-egg to a normal egg.)
After Jim's explanation (thanks!) I don't see a reason why develop-eggs couldn't also be shared... no, wait, perhaps I see one. Imagine project A using a develop-egg of a *branch* of zope.frobozz, while project B uses a develop-egg of the *trunk* of zope.frobozz. In that case /path/to/project-A/develop-eggs/zope.frobozz.egg-link will contain a link to ~/src/zope.frobozz/random-feature-branch, while /path/to/project-B/develop-eggs/zope.frobozz.egg-link would containt a link to ~/src/zope.frobozz/trunk. Same filename, different content -- ergo, develop-eggs cannot be shared between projects. I have a couple of source trees in my home directory that contain an actual example of this.
I kind of dislike having artifacts like these live in my version controlled source directory (well, there's bin too, but I can kind of live with that). What's the best way of doing that?
echo 'bin parts develop-eggs' > .csvignore svn propedit svn:ignore . bzr ignore bin parts develop-eggs or the equivalent for the VCS of your choice.
Actually, it's not so much 'bzr stat' that I care about since that's easily ignored.
(in which case your wording misled me ;)
It's 'ls' that I care about since that's tougher to ignore. I guess I could use dot-prefixes. ;)
Personally, I'm more bothered by the '*.egg-info' directories created by setuptools. Also, but to a somewhat lesser extent, the 'build' and 'dist' ones created by distutils. 'bin' and 'parts' are something I need to deal with rather often. I've grown to like bin/$randomscript, and while 'parts/$whatever/$somefile' is a bit too deep and nested, I appreciate the namespacing and explicitness ("this is a directory managed by buildout; it may be wiped out at any moment; don't touch it"). 'develop-eggs' is something that I'm not personally interested in and wouldn't mind hiding with a dot-prefix, but since I can't do that with 'build' and 'dist', I don't see much point. As long as it doesn't interfere with my tab-completion, I'm fine. Regards, Marius Gedminas -- BASIC: A programming language. Related to certain social diseases in that those who have it will not admit it in polite company.
On Jan 8, 2009, at 4:28 PM, Marius Gedminas wrote:
On Thu, Jan 08, 2009 at 04:12:11PM -0500, Barry Warsaw wrote:
On Jan 8, 2009, at 1:24 PM, Marius Gedminas wrote:
On Mon, Jan 05, 2009 at 03:02:24PM -0500, Barry Warsaw wrote:
I'm also wondering, does it make sense to put develop-eggs and parts in say ~/.buildout?
No, since those are specific to a particular project and shouldn't be shared.
Ok.
(Actually, I don't understand what 'develop-eggs' is for, other than that removing it tends to fix weird problems after switching from a develop-egg to a normal egg.)
After Jim's explanation (thanks!) I don't see a reason why develop- eggs couldn't also be shared... no, wait, perhaps I see one.
Imagine project A using a develop-egg of a *branch* of zope.frobozz, while project B uses a develop-egg of the *trunk* of zope.frobozz. In that case /path/to/project-A/develop-eggs/zope.frobozz.egg-link will contain a link to ~/src/zope.frobozz/random-feature-branch, while /path/to/project-B/develop-eggs/zope.frobozz.egg-link would containt a link to ~/src/zope.frobozz/trunk. Same filename, different content -- ergo, develop-eggs cannot be shared between projects.
I have a couple of source trees in my home directory that contain an actual example of this.
Right. That's the main reason they shouldn't be shared. My explanation was also incomplete. If you use the zc.recipe.egg:custom recipe to do a custom build of a binary, then the resulting binary egg is also put in develop-eggs because the egg depends on the buildout configuration. Jim -- Jim Fulton Zope Corporation
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On Jan 8, 2009, at 4:28 PM, Marius Gedminas wrote:
After Jim's explanation (thanks!) I don't see a reason why develop- eggs couldn't also be shared... no, wait, perhaps I see one.
Imagine project A using a develop-egg of a *branch* of zope.frobozz, while project B uses a develop-egg of the *trunk* of zope.frobozz. In that case /path/to/project-A/develop-eggs/zope.frobozz.egg-link will contain a link to ~/src/zope.frobozz/random-feature-branch, while /path/to/project-B/develop-eggs/zope.frobozz.egg-link would containt a link to ~/src/zope.frobozz/trunk. Same filename, different content -- ergo, develop-eggs cannot be shared between projects.
I have a couple of source trees in my home directory that contain an actual example of this.
That makes sense. I don't do this, but it definitely makes sense.
I kind of dislike having artifacts like these live in my version controlled source directory (well, there's bin too, but I can kind of live with that). What's the best way of doing that?
echo 'bin parts develop-eggs' > .csvignore svn propedit svn:ignore . bzr ignore bin parts develop-eggs or the equivalent for the VCS of your choice.
Actually, it's not so much 'bzr stat' that I care about since that's easily ignored.
(in which case your wording misled me ;)
Sorry ;)
It's 'ls' that I care about since that's tougher to ignore. I guess I could use dot-prefixes. ;)
Personally, I'm more bothered by the '*.egg-info' directories created by setuptools. Also, but to a somewhat lesser extent, the 'build' and 'dist' ones created by distutils.
Yeah, I hate all of those too!
'bin' and 'parts' are something I need to deal with rather often. I've grown to like bin/$randomscript, and while 'parts/$whatever/$somefile' is a bit too deep and nested, I appreciate the namespacing and explicitness ("this is a directory managed by buildout; it may be wiped out at any moment; don't touch it"). 'develop-eggs' is something that I'm not personally interested in and wouldn't mind hiding with a dot-prefix, but since I can't do that with 'build' and 'dist', I don't see much point. As long as it doesn't interfere with my tab- completion, I'm fine.
bin is a tough one. I love its convenience. parts I've never used. setup_requires dependencies also show up here (I always see the bzr egg because I setup-depend on setuptools_bzr). One other thing that bugs me about these is that it's not obvious from an ls what you can actually delete because it's an artifact of setuptools/buildout/etc and what is part of your package. I mean, you have to know that it's safe to delete bin, dist, and build right? setup.py has a 'clean' command but it's not enough. Maybe if buildout added a --clean option or something that would be better. What I really want is a simple way to revert a tree back to its pristine version-controlled layout, without all the build, develop, test artifacts. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin) iQCVAwUBSWZyAXEjvBPtnXfVAQKOYwP/Y3f/KD+/xXRzIP3TyRFUgtVywAfdnytU +H58rajSCt/WrYhwkLlFxBga/+/FQ7HqtpBQYZ7WEidh6FM2qjMmVGkVgVtOE3kQ umBgYc25lLytwj2nS05ZRANHDLYhWNbq8XMxrNnWLMPxxlT48ldrziBRv8MrKAX7 cBxHuKzJOAY= =21nV -----END PGP SIGNATURE-----
On Thu, Jan 08, 2009 at 04:37:05PM -0500, Barry Warsaw wrote:
One other thing that bugs me about these is that it's not obvious from an ls what you can actually delete because it's an artifact of setuptools/buildout/etc and what is part of your package. I mean, you have to know that it's safe to delete bin, dist, and build right?
Or you could see which files are ignored by your VCS. E.g. with bzr you'd do 'rm `bzr unknowns`', and I'm sure you could write a one-liner to work with 'svn st -v'. Or you could get a fresh checkout, or use svn export.
setup.py has a 'clean' command but it's not enough. Maybe if buildout added a --clean option or something that would be better. What I really want is a simple way to revert a tree back to its pristine version-controlled layout, without all the build, develop, test artifacts.
I'd say this is up to your version-control system to implement this, then. It doesn't feel right to put this into buildout. E.g. how would it know that it's OK to erase ./eggs, but not OK to erase ~/.buildout/my-shared-egg-cache/, if the difference is just an option in ~/.buildout/default.cfg? Sure, you could see which files are inside the tree and which ones are outside, but it rapidly descends into guessing. Which you should refuse the temptation to, according to Zen. Also, there are files littering your source tree that aren't tracked in a VCS, and weren't created by buildout (e.g. *.pyc). I don't think bin/buildout clean should remove files bin/buildout didn't create. Marius Gedminas -- Attempts to stick to simple on-or-off options lead to monsters like gcc, which now has so many flags that programmers are using genetic algorithms to explore them. -- http://www.third-bit.com/~gvwilson/xmlprog.html
participants (3)
-
Barry Warsaw -
Jim Fulton -
Marius Gedminas