[Python-Dev] [Distutils] PEP 376 - from PyPM's point of view

[Paul Moore]

2009/7/15 P.J. Eby <pje at telecommunity.com>:
> At 11:10 AM 7/15/2009 +0100, Paul Moore wrote:
>>
>> I propose that before the current prototype is turned into a final
>> (spec and) implementation, the PEP 302 extensions are extracted and
>> documented as an independent protocol, purely part of PEP 376. (This
>> *helps* implementers, as they can write support for, for example,
>> eggs, without needing to modify the existing egg importer).
>
> Btw, this is why setuptools chose to implement these things as adapters or
> generic functions in pkg_resources (and in the vestigial bits that were
> added to python 2.5's pkgutil).
>
> So as you can see, trying to solve these particular problems tends to lead
> to reinventing setuptools or at least portions thereof.  ;-)

Yes, that's the impression I'm getting :-)

And yet you seem resistant to reading any message into the fact that
people find it easier to solve the problems from scratch than to
extract the relevant parts of the existing code. To be explicit - it's
too flipping hard to identify, extract and tidy up the appropriate
bits of the setuptools code! (Disclaimer: I haven't looked at the code
for some time - if the style has changed utterly in the last 2 years
or so, so that it no longer resembles the style of the pkgutil code
you contributed to the stdlib, I'll retract that criticism of the
setuptools codebase).

>> But the
>> problem is that too many people dislike setuptools as it stands for it
>> to gain support. My understanding is that the current set of PEPs were
>> intended to be a stripped down, more generally acceptable subset of
>> setuptools. Let's keep them that way (and omit the complexities of
>> multi-version support).
>
> Even without multi-version support, the parts of PEP 376 that aren't about
> uninstallation are still reinventing chunks of pkg_resources.  Had
> pkg_resources been in the stdlib a couple years back (note that bugs and
> changes in it are still quite rare), the PEP 376 bits for pkgutil could have
> been focused strictly on uninstallation, and just reused pkg_resources API
> for finding distributions, reading metadata, getting version info, etc. etc.
>
> All that stuff is extremely stable code, very widely used for a very long
> time.  If politics is the only thing keeping it from being used, then that's
> a pretty sad statement about the community, given that pkg_resources is not
> to blame for 99% of what people complain about in relation to setuptools.
>  All pkg_resources does is find stuff (on request) and maybe add it to
> sys.path (on request), pull out data and metadata from distributions (on
> request), or set up namespace packages (on request).

I don't think it's quite "politics" as you describe it. Maybe
"marketing", if you want a simple tag for it.

The problem is that the setuptools "brand name" is associated with a
lot of baggage, that maybe it doesn't deserve, but equally no-one
seems interested in addressing. It's certainly not a technical
argument at this point. And yet you counter criticisms with technical
responses. But if I recall the history correctly, one of the initial
goals of setuptools was to become a de-facto standard precisely by
marketing and filling the ecological niche - a goal it has achieved
fantastically well. The fact that in doing so it alienated certain
groups is a more or less inevitable consequence.

My argument - which you may well disagree with - is that the
"setuptools ecosystem" (by which I mean
setuptools/easy_install/pkg_resources/eggs and the various subsystems
around these) is documented in a way which is:

- confusing to the casual user
- not at all modular (ie, it's impossible to easily pick out individual "bits")
- full of new concepts, so it's hard to find an easy place to start from

There's not much technically at issue here, it's all about presentation.

> So if politics demands that it be rejected by association with "setuptools",
> then just search-and-replace the API, PEP 8-ify it, call it something
> different, and lie to everyone about where it came from.  I won't tell if
> you won't.  ;-)

Unless the documentation and code is cleaned up, it'll still be
obvious from the style where it came from - that's the problem. Who's
going to maintain the code within the standard library, and are they
happy with the style of the existing code - even with the serial
numbers filed off?

And of course, someone has to do the clean-up. It seems to me that the
fact that people are more inclined to reinvent the code than to try to
understand the existing codebase and pick out the relevant bits, says
something important about how easy it would be to maintain the
existing code within the Python core...

Paul.