2009/7/15 P.J. Eby <pje@telecommunity.com>:

At 11:10 AM 7/15/2009 +0100, Paul Moore wrote:

...

I propose that before the current prototype is turned into a final (spec and) implementation, the PEP 302 extensions are extracted and documented as an independent protocol, purely part of PEP 376. (This *helps* implementers, as they can write support for, for example, eggs, without needing to modify the existing egg importer).

Btw, this is why setuptools chose to implement these things as adapters or generic functions in pkg_resources (and in the vestigial bits that were added to python 2.5's pkgutil).

So as you can see, trying to solve these particular problems tends to lead to reinventing setuptools or at least portions thereof.  ;-)

Yes, that's the impression I'm getting :-) And yet you seem resistant to reading any message into the fact that people find it easier to solve the problems from scratch than to extract the relevant parts of the existing code. To be explicit - it's too flipping hard to identify, extract and tidy up the appropriate bits of the setuptools code! (Disclaimer: I haven't looked at the code for some time - if the style has changed utterly in the last 2 years or so, so that it no longer resembles the style of the pkgutil code you contributed to the stdlib, I'll retract that criticism of the setuptools codebase).

...

But the problem is that too many people dislike setuptools as it stands for it to gain support. My understanding is that the current set of PEPs were intended to be a stripped down, more generally acceptable subset of setuptools. Let's keep them that way (and omit the complexities of multi-version support).

Even without multi-version support, the parts of PEP 376 that aren't about uninstallation are still reinventing chunks of pkg_resources.  Had pkg_resources been in the stdlib a couple years back (note that bugs and changes in it are still quite rare), the PEP 376 bits for pkgutil could have been focused strictly on uninstallation, and just reused pkg_resources API for finding distributions, reading metadata, getting version info, etc. etc.

All that stuff is extremely stable code, very widely used for a very long time.  If politics is the only thing keeping it from being used, then that's a pretty sad statement about the community, given that pkg_resources is not to blame for 99% of what people complain about in relation to setuptools.  All pkg_resources does is find stuff (on request) and maybe add it to sys.path (on request), pull out data and metadata from distributions (on request), or set up namespace packages (on request).

I don't think it's quite "politics" as you describe it. Maybe "marketing", if you want a simple tag for it. The problem is that the setuptools "brand name" is associated with a lot of baggage, that maybe it doesn't deserve, but equally no-one seems interested in addressing. It's certainly not a technical argument at this point. And yet you counter criticisms with technical responses. But if I recall the history correctly, one of the initial goals of setuptools was to become a de-facto standard precisely by marketing and filling the ecological niche - a goal it has achieved fantastically well. The fact that in doing so it alienated certain groups is a more or less inevitable consequence. My argument - which you may well disagree with - is that the "setuptools ecosystem" (by which I mean setuptools/easy_install/pkg_resources/eggs and the various subsystems around these) is documented in a way which is: - confusing to the casual user - not at all modular (ie, it's impossible to easily pick out individual "bits") - full of new concepts, so it's hard to find an easy place to start from There's not much technically at issue here, it's all about presentation.

So if politics demands that it be rejected by association with "setuptools", then just search-and-replace the API, PEP 8-ify it, call it something different, and lie to everyone about where it came from.  I won't tell if you won't.  ;-)

Unless the documentation and code is cleaned up, it'll still be obvious from the style where it came from - that's the problem. Who's going to maintain the code within the standard library, and are they happy with the style of the existing code - even with the serial numbers filed off? And of course, someone has to do the clean-up. It seems to me that the fact that people are more inclined to reinvent the code than to try to understand the existing codebase and pick out the relevant bits, says something important about how easy it would be to maintain the existing code within the Python core... Paul.

2009/7/15 P.J. Eby <pje@telecommunity.com>:

At 11:10 AM 7/15/2009 +0100, Paul Moore wrote:

...

I propose that before the current prototype is turned into a final (spec and) implementation, the PEP 302 extensions are extracted and documented as an independent protocol, purely part of PEP 376. (This *helps* implementers, as they can write support for, for example, eggs, without needing to modify the existing egg importer).

Btw, this is why setuptools chose to implement these things as adapters or generic functions in pkg_resources (and in the vestigial bits that were added to python 2.5's pkgutil).

So as you can see, trying to solve these particular problems tends to lead to reinventing setuptools or at least portions thereof.  ;-)

Yes, that's the impression I'm getting :-) And yet you seem resistant to reading any message into the fact that people find it easier to solve the problems from scratch than to extract the relevant parts of the existing code. To be explicit - it's too flipping hard to identify, extract and tidy up the appropriate bits of the setuptools code! (Disclaimer: I haven't looked at the code for some time - if the style has changed utterly in the last 2 years or so, so that it no longer resembles the style of the pkgutil code you contributed to the stdlib, I'll retract that criticism of the setuptools codebase).

...

But the problem is that too many people dislike setuptools as it stands for it to gain support. My understanding is that the current set of PEPs were intended to be a stripped down, more generally acceptable subset of setuptools. Let's keep them that way (and omit the complexities of multi-version support).

Even without multi-version support, the parts of PEP 376 that aren't about uninstallation are still reinventing chunks of pkg_resources.  Had pkg_resources been in the stdlib a couple years back (note that bugs and changes in it are still quite rare), the PEP 376 bits for pkgutil could have been focused strictly on uninstallation, and just reused pkg_resources API for finding distributions, reading metadata, getting version info, etc. etc.

All that stuff is extremely stable code, very widely used for a very long time.  If politics is the only thing keeping it from being used, then that's a pretty sad statement about the community, given that pkg_resources is not to blame for 99% of what people complain about in relation to setuptools.  All pkg_resources does is find stuff (on request) and maybe add it to sys.path (on request), pull out data and metadata from distributions (on request), or set up namespace packages (on request).

I don't think it's quite "politics" as you describe it. Maybe "marketing", if you want a simple tag for it. The problem is that the setuptools "brand name" is associated with a lot of baggage, that maybe it doesn't deserve, but equally no-one seems interested in addressing. It's certainly not a technical argument at this point. And yet you counter criticisms with technical responses. But if I recall the history correctly, one of the initial goals of setuptools was to become a de-facto standard precisely by marketing and filling the ecological niche - a goal it has achieved fantastically well. The fact that in doing so it alienated certain groups is a more or less inevitable consequence. My argument - which you may well disagree with - is that the "setuptools ecosystem" (by which I mean setuptools/easy_install/pkg_resources/eggs and the various subsystems around these) is documented in a way which is: - confusing to the casual user - not at all modular (ie, it's impossible to easily pick out individual "bits") - full of new concepts, so it's hard to find an easy place to start from There's not much technically at issue here, it's all about presentation.

So if politics demands that it be rejected by association with "setuptools", then just search-and-replace the API, PEP 8-ify it, call it something different, and lie to everyone about where it came from.  I won't tell if you won't.  ;-)

Unless the documentation and code is cleaned up, it'll still be obvious from the style where it came from - that's the problem. Who's going to maintain the code within the standard library, and are they happy with the style of the existing code - even with the serial numbers filed off? And of course, someone has to do the clean-up. It seems to me that the fact that people are more inclined to reinvent the code than to try to understand the existing codebase and pick out the relevant bits, says something important about how easy it would be to maintain the existing code within the Python core... Paul.