Mailman 3 July 2010 - Distutils-SIG

Re: [Distutils] How to force installing setuptools instead of distribute ?
by P.J. Eby 03 Jul '10

03 Jul '10

At 12:37 AM 7/2/2010 +0900, David Cournapeau wrote: >On Fri, Jul 2, 2010 at 12:18 AM, P.J. Eby <pje(a)telecommunity.com> wrote: > > > > > Did you try asking for an *exact* version of setuptools with '==' > (not using > > '>=' or leaving off a version)? > >Yes. The exact command I have tried is > >easy_install setuptools==0.6c11 Ouch. I was under the impression that it only masqueraded as setuptools when given a ranged request, not an exact request. That's unfortunate.

2 1

Packaging situation + mailing list rules
by Tarek Ziadé 03 Jul '10

03 Jul '10

Hello, >From time to time this mailing list is getting very unpleasant to work in because some old disagreements, and because some people are starting to get really nasty. Here's a reminder of the current packaging situation, and a few rules I suggest, for the benefit of all 1. Python <= 2.7 contains Distutils 2. Distutils is frozen, and won't evolve anymore 3. Setuptools is built on Distutils 4. Distribute forks Setuptools for various reasons. If you disagree with this choice, there's no need to send a mail here, because this is done. You can send me a private mail, but don't send a mail here because things are always getting nasty afterwards. Make sure to read the archives first, to understand why it was done. 5. Debian, Ubuntu and Fedora are using Distribute rather than Setuptools. If you find differences in the behavior, you can send a mail in this list, so we can fix Distribute. If you disagree with the distros choice, there's no need to send a mail here, because this is done. You can send *them* a mail, or tell them you disagree with this choice. Of course, the most effective behavior would be to work on having Distribute fixed if you find an issue, but that's up to you. 6. Distutils2 is being built, and has its own mailing list now. Distutils2 will replace Distutils in Python 3.2 or 3.3. It' s built on the various PEPs that were accepted lately, and any help is welcome.. The list is here: http://groups.google.fr/group/the-fellowship-of-the-packaging, and is moderated. 7. New members of the lists will not know the situation, so they won't be able to follow these rules of course. But people that are hanging around here for years can apply them. Of course this is just a series of suggestion. You can disagree with all these rules if you want, but I think they need to be applied in this mailing-list for the benefit of all. Thanks, Tarek -- Tarek Ziadé | http://ziade.org

5 16

Re: [Distutils] Packaging situation + mailing list rules
by P.J. Eby 02 Jul '10

02 Jul '10

At 04:13 PM 7/2/2010 +0200, Tarek Ziadé wrote: >On Fri, Jul 2, 2010 at 4:00 PM, P.J. Eby <pje(a)telecommunity.com> wrote: >[..] > > Isn't it interesting how these rules prohibit open disagreement > or criticism > > (or even discussion!) of distribute and related matters, but *not* > > setuptools? > >There's a huge gap between criticism + discussion, and the habitual flame >of distribute vs setuptools. I think you know what I mean. Since, as far as I can tell, David's are the first negative comments I've seen regarding distribute, the only "habitual" flame I'm aware of would be people spreading falsehoods about setuptools. I personally don't consider mere criticism of setuptools' or distribute's code or policies to be flaming. > > On a separate note, I'm curious why discussion of Distutils2 development is > > not in a formal Python SIG, such as the Distutils-SIG. > >To avoid such threads and flames. Vigorous discussion is a normal part of life in a Python SIG, or any standards-development process. Lots of people have said plenty of mean and nasty things about setuptools here (and to a lesser extent, people said them about WSGI when I started that effort), but I have never called for them to be silenced. The only thing I object to is people making statements that they know (or should know) are false, or to people speaking with flagrant disregard for the truth in an apparent effort to score political points. There is *huge* difference to me between, "Hey PJ, I hate setuptools and it sucks!" and "Nobody should use setuptools because it is unmaintained." Regarding the first one, I say, "Sorry to hear that, good luck to you with your future endeavors." The second one, however, is FUD and an unnecessary smear tactic. And I find it strange that many distribute supporters seem to prefer this approach over highlighting distribute's *actual* distinctions from setuptools, such as (not a complete list): * It has more tests * It runs on Python 3 * It supports 2.6+ "user" directories * It is released more frequently * We review patches more quickly * It's available in Mercurial, so you can more easily track local changes and participate in development (And even *I* agree that these are all benefits for many people.) However, perhaps it's a reflection of the fact that these benefits are NOT universally salient -- for many people, not one of these improvements is important enough to merit switching. So, if someone has other reasons than *user benefit* for wanting everyone else to switch, then perhaps they are forced to resort to FUD to motivate people to switch. And THAT is what I find distasteful. If you want people to switch because it is *actually* better for them, then more power to you. However, trying to persuade people to switch for *political* reasons -- reasons that will not provide any actual *benefit* to the switchers -- that's just disgusting. If people want to get creative in their presentations of the truth, please stick to exaggerating the advantages of distribute, rather than exaggerating the disadvantages of setuptools. The former is to be expected, the latter is unnecessary nastiness... especially since I bend over backwards *not* to say bad things about distribute, even when I'm more or less forced to comment on technical differences between the two. Indeed, I take especial care to treat them as mere stylistic differences, even when I think that distribute's choices will actually lead to problems for a given user's use case, simply because I don't want to deal with the flaming that ensues any time I say something that can remotely be construed as a criticism of distribute. (Consider the brouhaha that erupted the time I simply stated that if people wanted to upgrade to the latest setuptools, they would need to uninstall distribute first -- which is a mere technical fact caused by the way distribute operates!)

1 0

Re: [Distutils] How to force installing setuptools instead of distribute ?
by Tarek Ziadé 02 Jul '10

02 Jul '10

On Thu, Jul 1, 2010 at 5:20 PM, P.J. Eby <pje(a)telecommunity.com> wrote: > At 10:50 AM 7/1/2010 +0200, Tarek Ziadé wrote: >> >> It's precisely because Ubuntu is a good distribution that they decided >> to switch to distribute to get the most active project in it. > > Really? I would've thought that the most *stable* version of a project > would generally be the better choice for an operating system distribution. > ;-) It's a better choice than setuptools, which is unmaintained for two years with pending bugfixes. Stable != unmaintained ;)

3 5

Re: [Distutils] easy_install wrong download site preference
by P.J. Eby 02 Jul '10

02 Jul '10

At 01:33 AM 7/2/2010 +0300, anatoly techtonik wrote: >On Fri, Jul 2, 2010 at 12:10 AM, P.J. Eby <pje(a)telecommunity.com> wrote: > >> > > >> > It prefers newer packages, or, if the versions are the same, it prefers > >> > the shortest download URL. Ã In this case, the Google Code > url is shorter. > >> > >> That's illogical. Better prefer PyPI if versions are the same. > > > > The "shortest path" logic is there to avoid certain file recognition > > problems that occur without it. Â The special case of PyPI isn't special > > enough to break those rules. > >Although practicality beats purity. Can you list those "certain file >recognition problems"? I.e. Explicit is better than implicit. I have a vague recollection that it was Fredrik Lundh's website that sparked the original realization of the need for preferring shorter URLs, but I wouldn't swear to it. I'd have to dig through years of revision history to find the original change, assuming I documented it well enough. The choice of short paths over long was also intended to favor nearby files over further ones, and local paths over URLs. (All that being said, it's still fundamentally a heuristic, and not a very good one at that. But that doesn't automatically make any other heuristic *better*; this is one area where status quo bias is a good thing.) >That's why it should use the site where all filenames are Python >downloads if filenames are the same. And how would that work with all the PyPI clones, private indexes, etc.? > > No. Â You'd need to remove the current "home_page" setting, or point it > > elsewhere. > >That's very strange. Then what download_url is for? The home page and download URLs are simply treated as pages which may contain links to files, if they are not themselves links to files. That's the only special status they have. > >> Â (I understand that people do not want to touch setuptools code > >> anymore) > > > > That's not really the issue; the issue here is that package precedence is > > based on a stable comparison scheme, where it doesn't make sense > to give one > > location priority over another, as it will simply lead to someone else > > complaining about the changed behavior, because they were relying on a > > different URL having precedence under the current scheme. > >These rules need to be described first. What if somebody already broke >the proper order and now everybody suffers? If autodiscovery rules >were well described - it was possible to analyse them and propose more >intuitive approach. Then if "someone else" will attempt to complain - >you could send them to the PEP or another "how and why" document. http://peak.telecommunity.com/DevCenter/setuptools#making-your-package-avai… http://peak.telecommunity.com/DevCenter/EasyInstall#package-index-api >I thought it will raise the weight of those links if there could be a >rel="download" attribute. There is no "weighting" of links - what is weighted are distributions, and distribution objects only have their raw URL available as a basis for sorting once the version and archive type (the two higher-precedence attributes) are considered. The place where a URL was retrieved from is not tracked, and thus can't be used for sorting without a good chunk of refactoring... which refactoring would likely break tools that build on top of setuptools' PackageIndex class. In short, what you're asking for is a pretty major feature that would be difficult to implement in a way that would be guaranteed not to break other things.

1 0

How to force installing setuptools instead of distribute ?
by David Cournapeau 02 Jul '10

02 Jul '10

Hi, Ubuntu Lucid uses distribute instead of setuptools, and I cannot manage to use setuptools with virtualenv because of this. I upgraded to the last version of virtualenv, which claims to install setuptools by default, but I still get distribute instead, and I would guess this is because of Ubuntu using distribute, but who knows.... Is there a simple way to force virtualenv to install setuptools (the PJE version, *not* the distribute fork) ? cheers, David

5 14

Re: [Distutils] easy_install wrong download site preference
by P.J. Eby 02 Jul '10

02 Jul '10

At 11:36 PM 7/1/2010 +0300, anatoly techtonik wrote: >On Wed, Jun 30, 2010 at 7:54 PM, P.J. Eby <pje(a)telecommunity.com> wrote: > > > > It prefers newer packages, or, if the versions are the same, it > prefers the shortest download URL. Â In this case, the Google Code > url is shorter. > >That's illogical. Better prefer PyPI if versions are the same. The "shortest path" logic is there to avoid certain file recognition problems that occur without it. The special case of PyPI isn't special enough to break those rules. >PyPI is that page. Google Code URL homepage doesn't have any Python >related downloads at all. There isn't any way to know that from the filename. >What if we set download_url instead of >homepage back to PyPI page - will it satisfy setuptools as a quick >fix? No. You'd need to remove the current "home_page" setting, or point it elsewhere. > (I understand that people do not want to touch setuptools code >anymore) That's not really the issue; the issue here is that package precedence is based on a stable comparison scheme, where it doesn't make sense to give one location priority over another, as it will simply lead to someone else complaining about the changed behavior, because they were relying on a different URL having precedence under the current scheme. What's more, even if I made this change and released it immediately into the wild, you would *still* have this problem for the entire existing installed base, which is considerable. (Many people have not upgraded from setuptools 0.6c2 or 0.6c9, which are many years old.) And, the distribute fork would need to match the behavior as well, or there's that group of people who may still end up with the wrong file. Already, AFAICT, distribute has changed (at least in the repository) the path precedence rules in a way that is not consistent with the way setuptools does it, for both URLs *and* local file paths, so I can't really give any guidance as to what their version of easy_install will do -- it may do the "right thing" in this case, but if so, it's essentially by accident. (i.e., it won't necessarily do the right thing in other cases, since their changes weren't motivated by the same issue.) (One thing that I *could* do, would be to give precedence to links with #md5 tags -- this would automatically make PyPI (and PyPI-clone) links score higher, and would be less likely to introduce problems than trying to force recognition of PyPI specifically. But this would still have the problem of getting out into the field in a timely way.) > > (Note: you will have to go into PyPI's administration interface > and manually change the home page link for *all past versions* as > well, due to the way the /simple index works.) > >Where is the relevant code for this PyPI? I wonder why it didn't set >rel="download" for PyPI downloads if it should? That's not the problem either; easy_install is *finding* those links just fine; if you use "easy_install -vvn protobuf" you'll see it list both the PyPI tgz's and the Google Code URLs as candidates; it's just using the URL length as the tie-breaker.

2 1

Re: [Distutils] easy_install wrong download site preference
by P.J. Eby 02 Jul '10

02 Jul '10

At 09:59 AM 6/30/2010 +0300, anatoly techtonik wrote: >Hello, > >protobuf project have problems with installing via easy_install. One >part of problem is that the project provides versioned archives >unrelated to Python that `easy_install` threats as Python packages. >Corresponding package with the same version number was uploaded to >PyPI archive, but the easy_install still downloads archive from Google >Code. Why? I though easy_install should prefer PyPI package. It prefers newer packages, or, if the versions are the same, it prefers the shortest download URL. In this case, the Google Code url is shorter. >Is it possible to raise the priority of PyPI mirror for protobuf >project somehow? No. If (e.g.) http://pypi.python.org/packages/source/p/protobuf/protobuf-2.3.0.tar.gz and http://protobuf.googlecode.com/files/protobuf-2.3.0.tar.gz aren't equivalent files, they should not be named the same thing -- especially since this practice can confuse humans as well as easy_install. On a practical level, if it's too late for the Python project to use a different name, I would suggest changing the PyPI homepage links (for current and past releases) to point to a Python-specific project page, that does not contain links to download the generic, non-Python package. This will keep easy_install from considering them as candidates for downloading. (Note: you will have to go into PyPI's administration interface and manually change the home page link for *all past versions* as well, due to the way the /simple index works.)

2 1

Re: [Distutils] How to force installing setuptools instead of distribute ?
by P.J. Eby 02 Jul '10

02 Jul '10

At 12:53 AM 7/2/2010 +0900, David Cournapeau wrote: >This is issue 142 on bitbucket (distribute fork on Tarek account), and >there was a similar bug on setuptools issue tracker (submitted by >Zooko as well), but cannot find it ATM. http://bugs.python.org/setuptools/issue17 A fix was released in setuptools more than 8 months ago.

1 0

Re: [Distutils] How to force installing setuptools instead of distribute ?
by P.J. Eby 01 Jul '10

01 Jul '10

At 09:00 PM 7/1/2010 +0530, Zubin Mithra wrote: >My sincere apologies. >I was under the impression that it was not being maintained. It'd be >great if you could point me towards the main setuptools trunk. There's a link on the PyPI project page, as well as a link to the 0.6 stable branch. There's also a pointer to the bug tracker there. See http://pypi.python.org/pypi/setuptools >However, I have not worded this out on any other forum, so that >should'nt be a problem. :) Thanks. Not only is it not true that it's not maintained, but it's actually actively developed (albeit at a glacial pace). The most recent commits to the trunk this year are in relation to new features in progress for 0.7; I just don't make releases that often, and in fact I'm hesitating a bit about releasing 0.6final and 0.7a1 because that's when I expect certain parties will start cranking up the hate machine and FUDslinging again. :-(

2 1