Mailman 3 November 2009 - Distutils-SIG

Re: [Distutils] Bundling pkg_resources
by P.J. Eby 30 Nov '09

30 Nov '09

At 08:00 PM 11/30/2009 +0200, cool-RR wrote: >On Mon, Nov 30, 2009 at 7:57 PM, P.J. Eby ><<mailto:pje@telecommunity.com>pje(a)telecommunity.com> wrote: >At 02:09 PM 11/30/2009 +0000, Ram Rachum wrote: >I've asked before about bundling Distribute. But now I ask, is it possible to >pull out the `pkg_resources` module from the Distribute folder and bundle only >that with my project? > > >If your project is a standalone application with a completely >isolated sys.path, then yes, you can bundle it. If your project is >a library or shares sys.path directories with other libraries or >applications, then no, *do not bundle it*. > >(This is true even for the original pkg_resources, but it's doubly >true for Distribute's pkg_resources, as installing it may break a >setuptools-based installation's ability to upgrade setuptools.) > > >What I was thinking is to put it in its own package, and then import >it like `from my_package import pkg_resources`. > >Would that still be problematic? Not if your modules are the only ones using any pkg_resources APIs within a given program. But consider what happens if both your package and another package are calling 'require()' -- just to give a really simple example. You will have two working_set objects that don't agree with each other, or with sys.path. (Similar issues may also apply for other pkg_resources data structures and APIs.) Bundling pkg_resources is really not a good idea for anything but a 100% standalone application or similarly controlled environment, where you know that nothing else will be doing that kind of thing. (Bundling it alongside your setup.py to use during installation -- but not actually installing it -- would also be relatively safe.) If all you're worried about is that a user's version of setuptools might have some sort of bug in pkg_resources, you needn't be. It's the single most stable part of the setuptools code base, both in the senses of "infrequently changing" and "not a lot of bugs". Even if someone's using a version of pkg_resources that's say, 2 or 3 years old, you're not likely to see any bugs more severe than a spurious warning message that something might be being imported from more than one place.

2 1

Re: [Distutils] PEP 386 status - last round here ?
by P.J. Eby 30 Nov '09

30 Nov '09

At 09:15 PM 11/29/2009 +0100, Tarek Ziadé wrote: >2009/11/29 P.J. Eby <pje(a)telecommunity.com>: >[..] > > > > WSGI and setuptools have been widely adopted in spite of their > technical and > > ideological flaws, because they had good incentive engineering. > > > > Or, in other words, because practicality beats purity, every single time. > >Do you mean here that this independently-created standard, this good >incentive engeneering a.k.a. >Setuptools, is doomed not to evolve anymore ? e.g. to adapt its >standard to a common standard that is going to raise and be added in >stldib at some point ? I'm saying that ignoring backwards compatibility (as MAL and Ben have proposed) is bad incentive engineering. I don't have any disagreement with what you're doing wrt PEP 386, btw; we already had all the technical issue discussion necessary back when you first wrote it. I've just been chiming in on a couple of threads where people have proposed changes that would break backward compatibility and present user experience.

4 3

Re: [Distutils] Common version-comparison semantics for peace love and harmony
by P.J. Eby 29 Nov '09

29 Nov '09

At 09:41 AM 11/28/2009 +0100, Tarek Ziadé wrote: >That's completely wrong, the proposal is a benefit for all of us, >because it standardizes something that is already being done. You seem to have misunderstood me; I'm objecting to Ben Finney's "simple alphanumeric sort", not to PEP 386 in general. >PEP 386 propose a scheme to be adopted by developers or tools, but if some >people want to stick with their own internal version scheme for >development versions or post/pre release versions, they can do it >without any problem. And they don't have to follow >any PEP 386 convention for their internal work. This is not actually true, since developers working in a team situation can be sharing and building binary releases of these packages. >So you have two choices: >- an implicit, heuristic ordering (that's what is happening today) >- a explicit, documented ordering. that's the goal of PEP 386. Setuptools' version scheme *is* explicit and documented -- as you should know, since I pointed you to those docs when you were writing PEP 386.

2 1

Re: [Distutils] Common version-comparison semantics for peace love and harmony
by Laura Creighton 29 Nov '09

29 Nov '09

It occurs to me that this problem would go away if we had a way to ask, for any given version number, 'what was your creation date' and the sorting 'earlier' and 'later' by that date. Can somebody explain why we aren't doing this? Laura

3 2

Re: [Distutils] Common version-comparison semantics for peace love and harmony
by Laura Creighton 29 Nov '09

29 Nov '09

In a message of Sat, 28 Nov 2009 11:01:45 GMT, Floris Bruynooghe writes: >On Sat, Nov 28, 2009 at 10:50:36AM +0100, Laura Creighton wrote: >> But I think that it is the other way around ... what we want is a >> timestamp. The algorithm is for guessing which version is ealier >> in the absence of a timestamp. > >Does this work when you have a two maintenance branches releasing in >parallel? Consider this example: > >2009-01-01: 1.0 >2009-02-01: 1.1 >2009-02-20: 1.0.1 (1.0 maint branch) >2009-02-21: 1.1.1 (1.1 maint branch) > > >Regards >Floris Not if the only thing you sort on is the date -- which by the way should have hours and minutes as well -- because, as you say that is not enough to tell the 1.1 branch from the 1.0 branch. But once you have a name for a series of releases, then a timestamp can sort the series. Let us say that you now find a horrible bug and make: 2009-02-22-22:01: 1.0.2 2009-02-22-22:04: 1.1.2 Now I want to say 'requires this bugfix'. Right now I think that if I say requires 1.0.2 or later, then people with 1.1 will expect that they are ok, when they are not. Or am I misunderstanding? Laura

3 2

The "incompatible versions" problem
by Chris Withers 28 Nov '09

28 Nov '09

Hi All, Do you guys on this lists have any ideas how packages can indicate to package managers like setuptools could indicate that if they are required by another package, that requirement must be for greater than a certain version? This stems from the "backwards compatibility" discussion for zope.interface here: https://mail.zope.org/pipermail/zope-dev/2009-November/038417.html Off the top of my head, zope.component could spell it's requirement as: def setup( ... incompatible_with='<4.0' ) Whereby, setuptools would raise an exception if the following packages were installed: def setup( ... install_requires=['zope.component'] ) def setup( ... install_requires=['zope.component >= 3.4.1'] ) ...while it wouldn't for the following package: def setup( ... install_requires=['zope.component >= 4'] ) Ideas? cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

2 2

Re: [Distutils] Common version-comparison semantics for peace love and harmony
by Laura Creighton 28 Nov '09

28 Nov '09

In a message of Sat, 28 Nov 2009 10:27:14 +0100, Tarek Ziadé writes: >On Sat, Nov 28, 2009 at 7:31 AM, Laura Creighton <lac(a)openend.se> wrote: >> It occurs to me that this problem would go away if we had a way to >> ask, for any given version number, 'what was your creation date' and >> the sorting 'earlier' and 'later' by that date. Can somebody explain >> why we aren't doing this? > >You mean like a timestamp before or after the version ? > >I might be wrong but I think that would be similar to what RPM calls >an Epoch. A number that can be used to compare two packages when their >versions number don't follow the standard scheme anymore. But that's >just a fallback. >But for the sake of simplicity and standardization, this extra number >is avoided. > >Meaning that it would be better to define and use a standard for the >released packages, than introducing a timestamp and say: do whatever >you want with your version numbers. But I think that it is the other way around ... what we want is a timestamp. The algorithm is for guessing which version is ealier in the absence of a timestamp. >At some point, we all agree that MAJOR.MINOR.MICRO is an accepted >standard and we are arguing about pre/post/dev releases. We have no way to enforce this on the world. The arguments here may have gone away, but out in the wild, I think we need to accept that people are going to name their packages whatever they like. We can suggest in the strongest possible terms that they use this numbering convention, but it will still just be a suggestion. >My point is that we are able to define a public scheme for those too, >so all package installers share the same conventions. > >This doesn't imply that developers have to use it internally. It just >implies that, if the developer releases his package at some point to >PyPI, he will know *how* his version number will be sorted by package >managers, because it's documented in a PEP. e.g. a shared convention. It's not that I am 'anti-convention', it is just that I think that if we include a timestamp, and use that _first_ when trying to find out if a package is earlier than another package, then we will be able to interoperate with those people who out of ignorance or some business reason cannot respect the convention. Laura >Regards >Tarek

6 5

Re: [Distutils] Common version-comparison semantics for peace love and harmony
by Laura Creighton 28 Nov '09

28 Nov '09

In a message of Sat, 28 Nov 2009 21:34:00 +0900, David Cournapeau writes: >On Sat, Nov 28, 2009 at 9:22 PM, Laura Creighton <lac(a)openend.se> wrote: >> In a message of Sat, 28 Nov 2009 11:01:45 GMT, Floris Bruynooghe writes >: >>>On Sat, Nov 28, 2009 at 10:50:36AM +0100, Laura Creighton wrote: >>>> But I think that it is the other way around ... what we want is a >>>> timestamp. Â The algorithm is for guessing which version is ealier >>>> in the absence of a timestamp. >>> >>>Does this work when you have a two maintenance branches releasing in >>>parallel? Â Consider this example: >>> >>>2009-01-01: 1.0 >>>2009-02-01: 1.1 >>>2009-02-20: 1.0.1 Â Â (1.0 maint branch) >>>2009-02-21: 1.1.1 Â Â (1.1 maint branch) >>> >>> >>>Regards >>>Floris >> >> Not if the only thing you sort on is the date -- which by the way shoul >d >> have hours and minutes as well -- because, as you say that is not enoug >h >> to tell the 1.1 branch from the 1.0 branch. Â But once you have a name fo >r >> a series of releases, then a timestamp can sort the series. >> >> Let us say that you now find a horrible bug and make: >> >> 2009-02-22-22:01: 1.0.2 >> 2009-02-22-22:04: 1.1.2 >> >> Now I want to say 'requires this bugfix'. Right now I think that if >> I say requires 1.0.2 or later, then people with 1.1 will expect that >> they are ok, when they are not. Â Or am I misunderstanding? > >This is a slippery road. You will always find cases where any version >scheme will fail if you start caring about those issues through >version schemes only. In your example, that would be if different >micro releases are not synchronized, which happens quite often in my >experience. > >cheers, > >David Yes, this is why I am coming to the conclusion that we are asking too much of a versioning scheme. Thus I think we need, instead, a way to specify two things. First of all Requires-Dist: needs to be able to specify multiple branches. And second of all we need to get a unique name for a branch. But once we have that, we can work on timestamps to find 'later' -- and we won't have to worry if people need to make 3.0.gamma or 3.0.c4 in the future because their beta went really badly, or something. And, all the while, there will be the strict naming convention for those of us who like such things, and I am one of them. I just want to be able to interoperate with people who don't. Laura

2 1

Re: [Distutils] PEP 386 status - last round here ?
by Laura Creighton 28 Nov '09

28 Nov '09

Let us say a package forks. Or there are two packages that provide much similar functionality. I want to say that my package depends on having one or the other of the packages. How would I do that? A concrete example would be: requires nose 0.11.1 or greater OR py.test 1.1.1 or greater (because what I have written is a plugin that works with either, but requires a py.test or a nose that is recent enough to have the plugin architecture) Laura

1 0

zc.buildout usage in the real world
by Alex Rades 27 Nov '09

27 Nov '09

Hi, I'm currently working on a project based on django, together with a dozen of external libraries. Often I am in one of these scenarios: - I discover bugs in an external libraries, and i fix them, but the upstream author is busy and I need to deploy the fix ASAP. So I produce a patch. - I have to install a particular revision of an external library from git/svn/hg The approach I've used so far has been setting up a private pypi-compatible repository, and recreate packages which fit my needs. But it really feels like killing an ant with a sledgehammer. I don't understand if there is a better way to: - Patch packages before installing - Fetch packages via git/svn/hg or via direct http link I'm currently using the *great* minitage.recipe.scripts recipe to do all of this, and infact i can do something like: ---------------------------- [complex-deps] interpreter = python recipe = minitage.recipe.scripts urls = http://media.djangoproject.com/releases/1.1.1/Django-1.1.1.tar.gz http://github.com/robhudson/django-debug-toolbar/commit/f36b72655349cf3b8e9… Django-1.1.1-patches = ${buildout:directory}/patches/django-admin-fixes.diff debug_toolbar-patches = ${buildout:directory}/patches/debug_toolbar_sqlpanel.diff ----------------------------- Which is very very handy, it handles all my use case in a compact and nice way. The minitage.recipe.scripts is nice but is not widely used, so I'm a bit worried about its future and possible breakages. You know - when you work on a large project you have to think hard about your dependencies. It's like getting married. So the question is: Are there alternative solutions? Am I the only one with such needs? Thank you

4 6

Distutils-SIG November 2009