Proposal for incorporating buildout-versions on buildout (Re: Better version pinning in buildout (buildout-versions))
On Sat, Jan 5, 2013 at 5:47 PM, Jim Fulton <jim@zope.com> wrote: ...
I propose that buildout-versions get incorporated into buildout in the following way:
OK, proposal 1 wasn't accepted. Here's another stab: Proposal 2 ---------- 1. The ``allow-picked-versions`` option gets a new allowed value of ``show``. if there are unpicked versions and this option is set to ``show``, then picked/unpinned versions are reported in a way suitable for copying into a versions section, presumably with the same format used by buildout-versions today. 2. New buildout option: ``update-versions-file``. This takes a path (relative to buildout directory) of a file to update with any unpinned versions (in a manner roughly the same as buildout-versions does today). 3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now. 4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version. 5. In buildout 2, The default value of the versions option will be "versions", rather than being unset. This will allow users to omit:: version = versions from their buildout section. 6. To make it a little easier to supply buildout versions on the command line, make buildout the default section for command-line options, so:: update-versions-file=versions.cfg or:: allow-picked-versions=show would be allowed. (They are rejected now.) Thoughts? Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On 1/12/2013 11:18 AM, Jim Fulton wrote:
On Sat, Jan 5, 2013 at 5:47 PM, Jim Fulton <jim@zope.com> wrote: ...
I propose that buildout-versions get incorporated into buildout in the following way:
OK, proposal 1 wasn't accepted. Here's another stab:
Proposal 2 ----------
1. The ``allow-picked-versions`` option gets a new allowed value of ``show``. if there are unpicked versions and this option is set to ``show``, then picked/unpinned versions are reported in a way suitable for copying into a versions section, presumably with the same format used by buildout-versions today.
2. New buildout option: ``update-versions-file``. This takes a path (relative to buildout directory) of a file to update with any unpinned versions (in a manner roughly the same as buildout-versions does today).
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
5. In buildout 2, The default value of the versions option will be "versions", rather than being unset. This will allow users to omit::
version = versions
from their buildout section.
6. To make it a little easier to supply buildout versions on the command line, make buildout the default section for command-line options, so::
update-versions-file=versions.cfg
or::
allow-picked-versions=show
would be allowed. (They are rejected now.)
Thoughts?
+1. This would work perfectly for me. -- Eric.
On 2013-01-12 16:18:36 +0000, Jim Fulton said:
On Sat, Jan 5, 2013 at 5:47 PM, Jim Fulton <jim@zope.com> wrote: ...
I propose that buildout-versions get incorporated into buildout in the following way:
OK, proposal 1 wasn't accepted. Here's another stab:
Proposal 2 ----------
1. The ``allow-picked-versions`` option gets a new allowed value of ``show``. if there are unpicked versions and this option is set to ``show``, then picked/unpinned versions are reported in a way suitable for copying into a versions section, presumably with the same format used by buildout-versions today.
So the possible values becomes: true, false, show (which is true & verbose)
2. New buildout option: ``update-versions-file``. This takes a path (relative to buildout directory) of a file to update with any unpinned versions (in a manner roughly the same as buildout-versions does today).
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
Did somebody ask for this? I tend to think of develop eggs as "real" eggs. So if I want to resolve a conflict I'll just edit the develop-egg (presumably by changing the version of the develop egg in the case of a non-develop-egg that requires a particular version of a develop egg.)
5. In buildout 2, The default value of the versions option will be "versions", rather than being unset. This will allow users to omit::
version = versions
from their buildout section.
\o/
6. To make it a little easier to supply buildout versions on the command line, make buildout the default section for command-line options, so::
update-versions-file=versions.cfg
or::
allow-picked-versions=show
would be allowed. (They are rejected now.)
So that means I can pass in foo=bar and it will be applied to the buildout section? How about allowing parameter/values to be applied to any section and defaulting to buildout? E.g.: $ buildout buildout:update-versions-file=versions.cfg Is this same as: $ buildout update-versions-file=versions.cfg But I can also do: $ buildout foo:bar=baz To set parameter 'bar' with value 'baz' in section 'foo'. In any event, proposal 2 sounds "close enough" to me so: +1.
Thoughts?
Jim
-- Alex Clark · https://www.gittip.com/aclark4life/
On Sat, Jan 12, 2013 at 03:42:48PM -0500, Alex Clark wrote:
On 2013-01-12 16:18:36 +0000, Jim Fulton said:
On Sat, Jan 5, 2013 at 5:47 PM, Jim Fulton <jim@zope.com> wrote: ...
I propose that buildout-versions get incorporated into buildout in the following way:
OK, proposal 1 wasn't accepted. Here's another stab:
Proposal 2 ----------
1. The ``allow-picked-versions`` option gets a new allowed value of ``show``. if there are unpicked versions and this option is set to ``show``, then picked/unpinned versions are reported in a way suitable for copying into a versions section, presumably with the same format used by buildout-versions today.
So the possible values becomes: true, false, show (which is true & verbose)
2. New buildout option: ``update-versions-file``. This takes a path (relative to buildout directory) of a file to update with any unpinned versions (in a manner roughly the same as buildout-versions does today).
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
Did somebody ask for this?
I believe I mentioned this. I used to trip on this gotcha practically every time: - work on package foo that depends on bar - discover a bug in bar that manifests when I use it from foo - check out bar from svn trunk - add a 'mg.cfg' in foo's source tree with [buildout] extends = buildout.cfg develop = ../bar - bin/buildout -c mg.cfg - try some import pdb; pdb.set_trace() or debug prints in ../bar/src/..., run a project in foo, wonder why the breakpoints/debug prints won't work, check bin/runfoo, see ~/.buildout/eggs/bar-1.2.3.egg in there, realize what's the matter - edit mg.cfg again, add [versions] bar = - run bin/buildout -c mg.cfg again, continue debugging. It's an unnecessary speedbump.
6. To make it a little easier to supply buildout versions on the command line, make buildout the default section for command-line options, so::
update-versions-file=versions.cfg
or::
allow-picked-versions=show
would be allowed. (They are rejected now.)
So that means I can pass in foo=bar and it will be applied to the buildout section? How about allowing parameter/values to be applied to any section
I believe that's already allowed:
$ buildout foo:bar=baz
To set parameter 'bar' with value 'baz' in section 'foo'.
Marius Gedminas -- Life begins when you can spend your spare time programming instead of watching television. -- Cal Keegan
On 13/01/2013 03:43, Marius Gedminas wrote:
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
Did somebody ask for this?
I believe I mentioned this.
I used to trip on this gotcha practically every time:
- work on package foo that depends on bar - discover a bug in bar that manifests when I use it from foo - check out bar from svn trunk - add a 'mg.cfg' in foo's source tree with [buildout] extends = buildout.cfg develop = ../bar - bin/buildout -c mg.cfg - try some import pdb; pdb.set_trace() or debug prints in ../bar/src/..., run a project in foo, wonder why the breakpoints/debug prints won't work, check bin/runfoo, see ~/.buildout/eggs/bar-1.2.3.egg in there, realize what's the matter - edit mg.cfg again, add [versions] bar = - run bin/buildout -c mg.cfg again, continue debugging.
It's an unnecessary speedbump.
Yeah, I thought it might be this ;-) I don't know how hard it'd be top implement but in an ideal world I'd prefer to see develop eggs trump any other sources but still have version requirements checked. An example of why would be as above, except that I check out the wrong branch (trunk versus the major version the project should be using), fix the bug, check in, release, remove develop line and get annoyed that I still have the problem. Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Tue, Jan 15, 2013 at 1:22 AM, Chris Withers <chris@python.org> wrote:
On 13/01/2013 03:43, Marius Gedminas wrote:
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
Did somebody ask for this?
I believe I mentioned this.
I used to trip on this gotcha practically every time:
- work on package foo that depends on bar - discover a bug in bar that manifests when I use it from foo - check out bar from svn trunk - add a 'mg.cfg' in foo's source tree with [buildout] extends = buildout.cfg develop = ../bar - bin/buildout -c mg.cfg - try some import pdb; pdb.set_trace() or debug prints in ../bar/src/..., run a project in foo, wonder why the breakpoints/debug prints won't work, check bin/runfoo, see ~/.buildout/eggs/bar-1.2.3.egg in there, realize what's the matter - edit mg.cfg again, add [versions] bar = - run bin/buildout -c mg.cfg again, continue debugging.
It's an unnecessary speedbump.
Yeah, I thought it might be this ;-) I don't know how hard it'd be top implement but in an ideal world I'd prefer to see develop eggs trump any other sources but still have version requirements checked.
That's the case today. So why did you change this in buildout-versions? (Or did you?) The buildout-versions docs only say that unpinned versions aren't reported for develop-eggs. They don't actually say that version requirements are ignored for develop eggs. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On 15/01/2013 10:36, Jim Fulton wrote:
Yeah, I thought it might be this ;-) I don't know how hard it'd be top implement but in an ideal world I'd prefer to see develop eggs trump any other sources but still have version requirements checked.
That's the case today.
I don't believe it is, if you have: [buildout] develop = /checkouts/somepackage.2.0.0 versions = versions [versions] somepackage = 1.4.2 ...I've always found that /checkouts/somepackage.2.0.0 isn't on the python path but that for version 1.4.2 is.
So why did you change this in buildout-versions? (Or did you?)
Certainly not intentionally ;-)
The buildout-versions docs only say that unpinned versions aren't reported for develop-eggs. They don't actually say that version requirements are ignored for develop eggs.
Yep, these are both correct. The hacks that buildout-versions inherits from buildout.dumppickedversions means that versions for develop eggs never showed up. I would have loved to fix that but ran out of energy. Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Tue, Jan 15, 2013 at 5:45 AM, Chris Withers <chris@python.org> wrote:
On 15/01/2013 10:36, Jim Fulton wrote:
Yeah, I thought it might be this ;-) I don't know how hard it'd be top implement but in an ideal world I'd prefer to see develop eggs trump any other sources but still have version requirements checked.
That's the case today.
I don't believe it is, if you have:
[buildout] develop = /checkouts/somepackage.2.0.0 versions = versions
[versions] somepackage = 1.4.2
...I've always found that /checkouts/somepackage.2.0.0 isn't on the python path but that for version 1.4.2 is.
Right, meaning the version requirement *is* checked. I guess I don't understand: "I don't know how hard it'd be top implement but in an ideal world I'd prefer to see develop eggs trump any other sources but still have version requirements checked."
So why did you change this in buildout-versions? (Or did you?)
Certainly not intentionally ;-)
The buildout-versions docs only say that unpinned versions aren't reported for develop-eggs. They don't actually say that version requirements are ignored for develop eggs.
Yep, these are both correct. The hacks that buildout-versions inherits from buildout.dumppickedversions means that versions for develop eggs never showed up. I would have loved to fix that but ran out of energy.
What would you like to fix? Can you state how you would like it to behave? (I'm still of the opinion that Martijn and Marius, and probably others, are right that develop eggs should be used even if they don't satisfy version requirements.) Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On 1/15/2013 6:30 AM, Jim Fulton wrote:
On Tue, Jan 15, 2013 at 5:45 AM, Chris Withers <chris@python.org> wrote:
On 15/01/2013 10:36, Jim Fulton wrote:
Yeah, I thought it might be this ;-) I don't know how hard it'd be top implement but in an ideal world I'd prefer to see develop eggs trump any other sources but still have version requirements checked.
That's the case today.
I don't believe it is, if you have:
[buildout] develop = /checkouts/somepackage.2.0.0 versions = versions
[versions] somepackage = 1.4.2
...I've always found that /checkouts/somepackage.2.0.0 isn't on the python path but that for version 1.4.2 is.
Right, meaning the version requirement *is* checked.
I guess I don't understand: "I don't know how hard it'd be top implement but in an ideal world I'd prefer to see develop eggs trump any other sources but still have version requirements checked."
I think it means that the develop egg is the only version that's considered. If it doesn't match the version requirement, it's an error. -- Eric.
Hi, On Tue, Jan 15, 2013 at 10:21 AM, Eric V. Smith <eric@trueblade.com> wrote:
[...]
I guess I don't understand: "I don't know how hard it'd be top implement but in an ideal world I'd prefer to see develop eggs trump any other sources but still have version requirements checked."
I think it means that the develop egg is the only version that's considered. If it doesn't match the version requirement, it's an error.
+1 "In the face of ambiguity, refuse the temptation to guess." It seems most people is in agreement with: * If there is a "develop" distribution, no other version of that distribution should be fetched and used. I imagine this could translate to: * The version from the "develop" distro should override the any "[versions]" definition * If the "develop" distro version conflicts with required versions from other distros, (but not with the [versions] specification) then fail w/ a clear message instead of fetching a non-develop distro: - "distribution foo.bar 0.5.3 requires baz.fred <= 0.17.0, but development distribution at 'src/baz.fred' has version 0.18.0" (The case where the a version conflict with [versions] already fails). BTW, the "mr.developer" extension does the above partially, by blanking any [versions] definition of a package in development. I guess the behaviour above could be achieved if mr.develop effectively put the version of development distro into the [version] section, instead of just blanking it. Cheers, Leo
On 15/01/2013 12:21, Eric V. Smith wrote:
...I've always found that /checkouts/somepackage.2.0.0 isn't on the python path but that for version 1.4.2 is.
Right, meaning the version requirement *is* checked.
I guess I don't understand: "I don't know how hard it'd be top implement but in an ideal world I'd prefer to see develop eggs trump any other sources but still have version requirements checked."
I think it means that the develop egg is the only version that's considered. If it doesn't match the version requirement, it's an error.
Yep, that's exactly what I meant, and a very succinct way of putting it too :-) Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Tue, Jan 15, 2013 at 12:21 PM, Eric V. Smith <eric@trueblade.com> wrote:
On 1/15/2013 6:30 AM, Jim Fulton wrote:
On Tue, Jan 15, 2013 at 5:45 AM, Chris Withers <chris@python.org> wrote:
On 15/01/2013 10:36, Jim Fulton wrote:
Yeah, I thought it might be this ;-) I don't know how hard it'd be top implement but in an ideal world I'd prefer to see develop eggs trump any other sources but still have version requirements checked.
That's the case today.
I don't believe it is, if you have:
[buildout] develop = /checkouts/somepackage.2.0.0 versions = versions
[versions] somepackage = 1.4.2
...I've always found that /checkouts/somepackage.2.0.0 isn't on the python path but that for version 1.4.2 is.
Right, meaning the version requirement *is* checked.
I guess I don't understand: "I don't know how hard it'd be top implement but in an ideal world I'd prefer to see develop eggs trump any other sources but still have version requirements checked."
I think it means that the develop egg is the only version that's considered. If it doesn't match the version requirement, it's an error.
That sounds like a good way to balance the possibility that the develop egg isn't being used and it is not immediately apparent why not and the possibility that your stated version requirements are silently not being met. -- Benji York
On Sat, Jan 12, 2013 at 3:42 PM, Alex Clark <aclark@aclark.net> wrote: ...
2. New buildout option: ``update-versions-file``. This takes a path (relative to buildout directory) of a file to update with any unpinned versions (in a manner roughly the same as buildout-versions does today).
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
Did somebody ask for this? I tend to think of develop eggs as "real" eggs. So if I want to resolve a conflict I'll just edit the develop-egg (presumably by changing the version of the develop egg in the case of a non-develop-egg that requires a particular version of a develop egg.)
Well, theres: http://packages.python.org/buildout-versions/use.html#development-packages which, rereading it, isn't quite the same thing. Martijn Faassen used to argue with me about this, that if you're developing something in a buildout, it should be used. I've come to think he was right. A challenge is that eggs are often classified as "develop" eggs by setuptools even though they aren't used for development. Eggs installed via system packagers are often installed this way. Thus, the requirement that only eggs that show up in a buildout's develop-eggs directory should be used even if they are pinned. ...
6. To make it a little easier to supply buildout versions on the command line, make buildout the default section for command-line options, so::
update-versions-file=versions.cfg
or::
allow-picked-versions=show
would be allowed. (They are rejected now.)
So that means I can pass in foo=bar and it will be applied to the buildout section? How about allowing parameter/values to be applied to any section and defaulting to buildout? E.g.:
That's what I'm proposing. You can currently supply an option in any section with sectionname:optionname=value. The proposal is to allow the section name to default to buildout.
$ buildout buildout:update-versions-file=versions.cfg
Is this same as:
$ buildout update-versions-file=versions.cfg
Yes
But I can also do:
$ buildout foo:bar=baz
To set parameter 'bar' with value 'baz' in section 'foo'.
You can do this today. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
Op 12-01-13 17:18, Jim Fulton schreef:
On Sat, Jan 5, 2013 at 5:47 PM, Jim Fulton <jim@zope.com> wrote: ....
I propose that buildout-versions get incorporated into buildout in the following way:
OK, proposal 1 wasn't accepted. Here's another stab:
Proposal 2 ----------
1. The ``allow-picked-versions`` option gets a new allowed value of ``show``. if there are unpicked versions and this option is set to ``show``, then picked/unpinned versions are reported in a way suitable for copying into a versions section, presumably with the same format used by buildout-versions today.
2. New buildout option: ``update-versions-file``. This takes a path (relative to buildout directory) of a file to update with any unpinned versions (in a manner roughly the same as buildout-versions does today).
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
I'm not sure about this one. Is the develop-eggs directory always cleaned up during a buildout run so only the eggs in the 'develop' option are listed there? If old eggs are kept there, this may lead to unexpected results, where old developed eggs are still used even when the buildout config does not mention them. If an egg is explicitly in the 'develop' option, then I would be +1 on favouring that over a pinned version. This indeed trips me up sometimes in current buildout 1, so this could help.
5. In buildout 2, The default value of the versions option will be "versions", rather than being unset. This will allow users to omit::
version = versions
from their buildout section.
6. To make it a little easier to supply buildout versions on the command line, make buildout the default section for command-line options, so::
update-versions-file=versions.cfg
or::
allow-picked-versions=show
would be allowed. (They are rejected now.)
Thoughts?
+1 -- Maurits van Rees: http://maurits.vanrees.org/ Zest Software: http://zestsoftware.nl
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 01/12/2013 04:53 PM, Maurits van Rees wrote:
Is the develop-eggs directory always cleaned up during a buildout run so only the eggs in the 'develop' option are listed there?
Yes. See: https://github.com/buildout/buildout/blob/master/src/zc/buildout/buildout.py... Tres. - -- =================================================================== Tres Seaver +1 540-429-0999 tseaver@palladion.com Palladion Software "Excellence by Design" http://palladion.com -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with undefined - http://www.enigmail.net/ iEYEARECAAYFAlDx5XgACgkQ+gerLs4ltQ7KRgCfSpy6MMfZtlqOa5JNhOTH5opj lJ0An2qCMPazGZIG3YuVeFSfaE2fZpwG =PEaD -----END PGP SIGNATURE-----
On Sat, Jan 12, 2013 at 4:53 PM, Maurits van Rees <m.van.rees@zestsoftware.nl> wrote:
Op 12-01-13 17:18, Jim Fulton schreef:
...
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
I'm not sure about this one. Is the develop-eggs directory always cleaned up during a buildout run so only the eggs in the 'develop' option are listed there?
Yes, or at least it's supposed to. Sometimes, the cleanup fails, In my experience this failure is very rare. ...
If an egg is explicitly in the 'develop' option, then I would be +1 on favouring that over a pinned version.
You don't specify eggs in develop options, you specify setup scripts. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On Sat, Jan 12, 2013 at 11:18:36AM -0500, Jim Fulton wrote:
On Sat, Jan 5, 2013 at 5:47 PM, Jim Fulton <jim@zope.com> wrote: ...
I propose that buildout-versions get incorporated into buildout in the following way:
OK, proposal 1 wasn't accepted. Here's another stab:
Proposal 2 ----------
1. The ``allow-picked-versions`` option gets a new allowed value of ``show``. if there are unpicked versions and this option is set to ``show``, then picked/unpinned versions are reported in a way suitable for copying into a versions section, presumably with the same format used by buildout-versions today.
+0.5 (the missing 0.5 is for aesthetic reasons, but I have no better suggestions at the moment, and besides bikeshedding on this would be counter-productive)
2. New buildout option: ``update-versions-file``. This takes a path (relative to buildout directory) of a file to update with any unpinned versions (in a manner roughly the same as buildout-versions does today).
+1
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
+1
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
+1 for the intent, -1 for the implementation. To me develop-eggs was always some kind of mystical implementation detail that sometimes broke things (leftover egg-link files even after I removed those names from my 'develop =' list and re-ran bin/buildout; which always occurred at a point in time when my internal stack was full and I couldn't investigate/file bugs and just rm-rf'ed develop-eggs to be able to continue). I suggest this instead: develop eggs explicitly listed in the [buildout] 'develop' options will be used even if their version conflicts with a pinned version.
5. In buildout 2, The default value of the versions option will be "versions", rather than being unset. This will allow users to omit::
version = versions
from their buildout section.
+1, assuming that was a misspelling of 'versions = versions'. (Corner case analysis: I expect that 'versions = ' will turn off version pinning. I've no intention of ever making use of that corner case)
6. To make it a little easier to supply buildout versions on the command line, make buildout the default section for command-line options, so::
update-versions-file=versions.cfg
or::
allow-picked-versions=show
would be allowed. (They are rejected now.)
+0.5 (gut feeling: I like this. I haven't had time to think about the possible consequences, so I'm withholding the other 0.5 for now.) (Aside: I always wanted buildout to have --newer and --not-newer, because I cannot for the life of me remember which is -n and which is -N. Being able to say bin/buildout newer=off would relieve that need considerably.) Marius Gedminas -- Beware of bugs in the above code; I have only proved it correct, not tried it. -- Donald Knuth
On Sat, Jan 12, 2013 at 7:38 PM, Marius Gedminas <marius@pov.lt> wrote: ...
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
+1 for the intent, -1 for the implementation.
To me develop-eggs was always some kind of mystical implementation detail that sometimes broke things (leftover egg-link files even after I removed those names from my 'develop =' list and re-ran bin/buildout; which always occurred at a point in time when my internal stack was full and I couldn't investigate/file bugs and just rm-rf'ed develop-eggs to be able to continue).
It may be that you're remember from before this was improved.
I suggest this instead: develop eggs explicitly listed in the [buildout] 'develop' options will be used even if their version conflicts with a pinned version.
The develop option doesn't list eggs. It lists paths to setup scripts. The only way we can tell what eggs a setup script installs is by watching the develop-eggs directory. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
Op 13-01-13 15:10, Jim Fulton schreef:
On Sat, Jan 12, 2013 at 7:38 PM, Marius Gedminas <marius@pov.lt> wrote: ....
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
+1 for the intent, -1 for the implementation.
To me develop-eggs was always some kind of mystical implementation detail that sometimes broke things (leftover egg-link files even after I removed those names from my 'develop =' list and re-ran bin/buildout; which always occurred at a point in time when my internal stack was full and I couldn't investigate/file bugs and just rm-rf'ed develop-eggs to be able to continue).
It may be that you're remember from before this was improved.
I suggest this instead: develop eggs explicitly listed in the [buildout] 'develop' options will be used even if their version conflicts with a pinned version.
The develop option doesn't list eggs. It lists paths to setup scripts. The only way we can tell what eggs a setup script installs is by watching the develop-eggs directory.
Ah, makes sense now. 'develop = src/my.package' probably leads to development of a package called my.package, but the name in setup.py may be totally different, so the develop-eggs dir seems the good place to look. And Tres pointed to the line in zc.buildout that cleans this directory up, so it looks safe to me. +1 -- Maurits van Rees: http://maurits.vanrees.org/ Zest Software: http://zestsoftware.nl
On 12/01/2013 16:18, Jim Fulton wrote:
On Sat, Jan 5, 2013 at 5:47 PM, Jim Fulton<jim@zope.com> wrote: ...
I propose that buildout-versions get incorporated into buildout in the following way:
OK, proposal 1 wasn't accepted. Here's another stab:
Proposal 2 ----------
1. The ``allow-picked-versions`` option gets a new allowed value of ``show``. if there are unpicked versions and this option is set to ``show``, then picked/unpinned versions are reported in a way suitable for copying into a versions section, presumably with the same format used by buildout-versions today.
2. New buildout option: ``update-versions-file``. This takes a path (relative to buildout directory) of a file to update with any unpinned versions (in a manner roughly the same as buildout-versions does today).
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
+1
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
This is the current behaviour you get with buildout-versions and it makes me a little nervous but I've never had the will to fight it and it is useful... But, most packages being developed do have a version in their setup.py (as an aside, what does buildout do with non-develop packages that don't specify a version?) and so it feels like the semantics would be cleaner if we just made everything work the same way when it came to versions.
5. In buildout 2, The default value of the versions option will be "versions", rather than being unset. This will allow users to omit::
version = versions
from their buildout section.
6. To make it a little easier to supply buildout versions on the command line, make buildout the default section for command-line options, so::
update-versions-file=versions.cfg
or::
allow-picked-versions=show
would be allowed. (They are rejected now.)
+1 Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
On Sat, Jan 12, 2013 at 11:18 AM, Jim Fulton <jim@zope.com> wrote:
On Sat, Jan 5, 2013 at 5:47 PM, Jim Fulton <jim@zope.com> wrote: ...
I propose that buildout-versions get incorporated into buildout in the following way:
OK, proposal 1 wasn't accepted. Here's another stab:
Proposal 2 ----------
1. The ``allow-picked-versions`` option gets a new allowed value of ``show``. if there are unpicked versions and this option is set to ``show``, then picked/unpinned versions are reported in a way suitable for copying into a versions section, presumably with the same format used by buildout-versions today.
2. New buildout option: ``update-versions-file``. This takes a path (relative to buildout directory) of a file to update with any unpinned versions (in a manner roughly the same as buildout-versions does today).
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
4. Change: develop eggs found in the buildout's develop-eggs directory will be used even if their version conflicts with a pinned version.
OK, so this turned out to be controversial. Some people want a variation on the current behavior (versions rule) but with an error if a develop version doesn't match. Others would like to use a develop egg even if it doesn't satisfy a version constraint from the versions section. So: Let's: - Exclude this from the proposal. - As a separate project (at some time), add an error check for develop eggs that are ignored because they don't satisfy a version constraint from the versions section. I'll consider this a bug fix, or at least not a backward incompatibility. - As yet another possible project, add an option to ignore version constraints (from the versions section) for develop eggs. Note that none of the above says anything about version requirements of individual distributions. It only applies to version requirements given in the versions section.
5. In buildout 2, The default value of the versions option will be "versions", rather than being unset. This will allow users to omit::
version = versions
from their buildout section.
I'll do this in 2.0.
6. To make it a little easier to supply buildout versions on the command line, make buildout the default section for command-line options, so::
update-versions-file=versions.cfg
or::
allow-picked-versions=show
would be allowed. (They are rejected now.)
I'll do this in 2.0 as well. So I think we're in agreement (as much as feasible). I'll do 5 and 6 in buildout 2.0. Hopefully, I can get a beta out this weekend. 1, 2, and 3 will be done when Chris (or someone else) makes time to work on a pull request. Perhaps this will be in buildout 2.1. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On 16-01-13 13:52, Jim Fulton wrote:
1, 2, and 3 will be done when Chris (or someone else) makes time to work on a pull request. Perhaps this will be in buildout 2.1.
I've made a branch where I noted the changes that have to be copied over (and adjusted) from buildout-versions. https://github.com/reinout/buildout/tree/reinout-versions Note that it doesn't strike me as a good idea to change "allow-picked-versions", which is now a nice true/false boolean, into a boolean that also accepts the "show" value. I can do it, but it looks dirty. Would an additional "show-picked-versions" be OK? Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On Wed, Jan 23, 2013 at 3:21 PM, Reinout van Rees <reinout@vanrees.org> wrote:
On 16-01-13 13:52, Jim Fulton wrote:
1, 2, and 3 will be done when Chris (or someone else) makes time to work on a pull request. Perhaps this will be in buildout 2.1.
I've made a branch where I noted the changes that have to be copied over (and adjusted) from buildout-versions. https://github.com/reinout/buildout/tree/reinout-versions
Note that it doesn't strike me as a good idea to change "allow-picked-versions", which is now a nice true/false boolean, into a boolean that also accepts the "show" value. I can do it, but it looks dirty.
Would an additional "show-picked-versions" be OK?
Sure. It means there are combinations that make no sense: show-picked-versions=true allow-picked-versions=false <shrug> I don't feel strongly about it. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On 16-01-13 13:52, Jim Fulton wrote:
On Sat, Jan 12, 2013 at 11:18 AM, Jim Fulton <jim@zope.com> wrote:
On Sat, Jan 5, 2013 at 5:47 PM, Jim Fulton <jim@zope.com> wrote: ...
I propose that buildout-versions get incorporated into buildout in the following way:
I've prepared a pull request: https://github.com/buildout/buildout/pull/46
1. The ``allow-picked-versions`` option gets a new allowed value of ``show``. if there are unpicked versions and this option is set to ``show``, then picked/unpinned versions are reported in a way suitable for copying into a versions section, presumably with the same format used by buildout-versions today.
Having a nice true/false boolean become a true/false/'show' hybrid: not too nice. I've added a show-picked-versions option (default: false). Set it to true and you get the picked versions at the end like buildout-versions would give you.
2. New buildout option: ``update-versions-file``. This takes a path (relative to buildout directory) of a file to update with any unpinned versions (in a manner roughly the same as buildout-versions does today).
Done.
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
Hm. I copy/pasted the buildout-versions code and worked from there. Buildout-versions simply looks at a 'python' entry in the [versions] part, so that's different from a python-version option. That's not too bad as it saves us another buildout option. But... do we want to change this?
1, 2, and 3 will be done when Chris (or someone else) makes time to work on a pull request. Perhaps this will be in buildout 2.1.
Should my pull request land in 2.0 instead of in 2.1? Alternative is that Chris accepts the buildout-versions pull request I prepared, but he's right (I think) in preferring it to land in buildout itself. (I myself cannot really use buildout 2 without the buildout-versions functionality either in buildout itself or in a new buildout-versions). Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On 25-01-13 21:53, Reinout van Rees wrote:
I've prepared a pull request: https://github.com/buildout/buildout/pull/46
I had to set up travis integration for my own branch as pull requests aren't tested. All tests pass: https://travis-ci.org/reinout/buildout/builds/4385204 Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On Fri, Jan 25, 2013 at 3:53 PM, Reinout van Rees <reinout@vanrees.org> wrote:
On 16-01-13 13:52, Jim Fulton wrote:
On Sat, Jan 12, 2013 at 11:18 AM, Jim Fulton <jim@zope.com> wrote:
On Sat, Jan 5, 2013 at 5:47 PM, Jim Fulton <jim@zope.com> wrote: ...
I propose that buildout-versions get incorporated into buildout in the following way:
I've prepared a pull request: https://github.com/buildout/buildout/pull/46
Thanks! ...
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
Hm. I copy/pasted the buildout-versions code and worked from there. Buildout-versions simply looks at a 'python' entry in the [versions] part, so that's different from a python-version option.
That's not too bad as it saves us another buildout option. But... do we want to change this?
I think so. Python itself is not a distutils distribution. This feels like mixing apples and oranges. Also, version entries can now have
=, >, <= and < modifiers, but you aren't supporting that for Python versions.
I'd rather make this a separate option. Also, the implementation shouldn't simply test whether the version is in the version string. It's too easy to get false matches. The implementation should parse the version (maybe as simple as splitting on dots) and check it against sys.version_info.
1, 2, and 3 will be done when Chris (or someone else) makes time to work on a pull request. Perhaps this will be in buildout 2.1.
Should my pull request land in 2.0 instead of in 2.1?
I'd rather get 2.0 out sooner. 2.1 doesn't have to lag far behind. If your pull request was perfect and I landed it right now, we'd need another beta. As it is, I think the handling of the Python version needs a little work.
Alternative is that Chris accepts the buildout-versions pull request I prepared, but he's right (I think) in preferring it to land in buildout itself.
The 2 aren't mutually exclusive.
(I myself cannot really use buildout 2 without the buildout-versions functionality either in buildout itself or in a new buildout-versions).
Ok, so you'd wait for 2.1, unless Chris releases a compatible buildout-versions. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On Mon, Jan 28, 2013 at 6:38 AM, Jim Fulton <jim@zope.com> wrote:
On Fri, Jan 25, 2013 at 3:53 PM, Reinout van Rees <reinout@vanrees.org> wrote:
...
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
Hm. I copy/pasted the buildout-versions code and worked from there. Buildout-versions simply looks at a 'python' entry in the [versions] part, so that's different from a python-version option.
That's not too bad as it saves us another buildout option. But... do we want to change this?
I think so. Python itself is not a distutils distribution. This feels like mixing apples and oranges. Also, version entries can now have
=, >, <= and < modifiers, but you aren't supporting that for Python versions.
I'd rather make this a separate option.
Also, the implementation shouldn't simply test whether the version is in the version string. It's too easy to get false matches. The implementation should parse the version (maybe as simple as splitting on dots) and check it against sys.version_info.
I should have mentioned that an option is to defer the python-version feature for later. Jim -- Jim Fulton http://www.linkedin.com/in/jimfulton Jerky is better than bacon! http://zo.pe/Kqm
On 28-01-13 12:50, Jim Fulton wrote:
On Mon, Jan 28, 2013 at 6:38 AM, Jim Fulton <jim@zope.com> wrote:
On Fri, Jan 25, 2013 at 3:53 PM, Reinout van Rees <reinout@vanrees.org> wrote:
...
3. New buildout option: ``python-version`` that restricts the Python version, with the same semantics as buildout-version provides now.
Hm. I copy/pasted the buildout-versions code and worked from there. Buildout-versions simply looks at a 'python' entry in the [versions] part, so that's different from a python-version option.
That's not too bad as it saves us another buildout option. But... do we want to change this?
I think so. Python itself is not a distutils distribution. This feels like mixing apples and oranges. Also, version entries can now have
=, >, <= and < modifiers, but you aren't supporting that for Python versions.
I'd rather make this a separate option.
Sounds logical.
Also, the implementation shouldn't simply test whether the version is in the version string. It's too easy to get false matches. The implementation should parse the version (maybe as simple as splitting on dots) and check it against sys.version_info.
Yeah, I saw that the implementation wasn't perfect. It looks at the whole string including "linux 2.1.xxx", so python=2 would work all the time. Copy/pasted from buildout-versions.
I should have mentioned that an option is to defer the python-version feature for later.
I'll take the python version out of my pull request for now and create a separate issue that points at the removed code. Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On 28-01-13 13:17, Reinout van Rees wrote:
I should have mentioned that an option is to defer the python-version feature for later.
I'll take the python version out of my pull request for now and create a separate issue that points at the removed code.
I've updated the pull request. Travis is running, https://travis-ci.org/reinout/buildout/builds/4428058 At the moment two of the four tests already passed. And I've added a new issue pointing at the removed code and with instructions to fix it: https://github.com/buildout/buildout/issues/50 Reinout -- Reinout van Rees http://reinout.vanrees.org/ reinout@vanrees.org http://www.nelen-schuurmans.nl/ "If you're not sure what to do, make something. -- Paul Graham"
On 28/01/2013 11:38, Jim Fulton wrote:
I think so. Python itself is not a distutils distribution.
...and isn't that a shame? :-( Wouldn't it be great if Python itself and all the packages in the standard library all had their own version numbers so you could upgrade them independently and have them evolve separately to the core? </rant>
Also, the implementation shouldn't simply test whether the version is in the version string. It's too easy to get false matches. The implementation should parse the version (maybe as simple as splitting on dots) and check it against sys.version_info.
Just to note that 'version in string' was a very careful choice as it lets you be as specific or general as you like. - buildout doesn't work with Python 3.x, just put '2.' in the option - want a very specific 3rd point release? put '3.3.1' - use a different python distro and need that? (my original use case!) put '2.7.2 |EPD 7.2-2 (32-bit)' sys.version_info doesn't cut it: buzzkill:testfixtures chris$ python Enthought Python Distribution -- www.enthought.com Version: 7.2-2 (32-bit) Python 2.7.2 |EPD 7.2-2 (32-bit)| (default, Sep 7 2011, 09:16:50) [GCC 4.0.1 (Apple Inc. build 5493)] on darwin Type "packages", "demo" or "enthought" for more information.
import sys sys.version_info sys.version_info(major=2, minor=7, micro=2, releaselevel='final', serial=0) sys.version '2.7.2 |EPD 7.2-2 (32-bit)| (default, Sep 7 2011, 09:16:50) \n[GCC 4.0.1 (Apple Inc. build 5493)]'
cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk
participants (10)
-
Alex Clark -
Benji York -
Chris Withers -
Eric V. Smith -
Jim Fulton -
Leonardo Rochael Almeida -
Marius Gedminas -
Maurits van Rees -
Reinout van Rees -
Tres Seaver