[Python-Dev] PEP 0365: Adding the pkg_resources module

[Phillip J. Eby]

At 11:44 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
>On 2007-05-21 22:48, Phillip J. Eby wrote:
> > At 08:56 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
> >> On 2007-05-21 20:01, Phillip J. Eby wrote:
> >> > At 06:28 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
> >> >> However, since this is not egg-specific it should probably be
> >> >> moved to pkgutil and get a separate PEP with detailed documentation
> >> >> (the link you provided doesn't really explain the concepts, reading
> >> >> the code helped a bit).
> >> >
> >> > That doesn't really make sense in the context of the current PEP,
> >> > though, which isn't to provide a general-purpose namespace package API;
> >> > it's specifically about adding an existing piece of code to the stdlib,
> >> > with its API intact.
> >>
> >> You seem to indicate that you're not up to discussing the concepts
> >> implemented by the module and *integrating* them with the Python
> >> stdlib.
> >
> > No, I'm saying something else.  I'm saying it:
> >
> > 1. has nothing to do with the PEP,
> > 2. isn't something I'm volunteering to do, and
> > 3. would only make sense to do as part of Python 3 stdlib
> > reorganization, if it were done at all.
>
>I don't understand that last part: how can adding a new module
>or set of modules require waiting for reorganization of the
>stdlib ?

I'm saying that an API reorganization that split up the pkg_resources 
API might be appropriate at that time, in much the same way that say, 
merging "dl" and "ctypes" (or dropping "dl") might take place during 
such a reorganization.  (And would thus go along with the 2to3 
conversion or whatever other mechanism will be used for porting 
Python 2 code to Python 3 code.)


>All I'm suggesting is to reorganize the code in pkg_resources.py
>a bit and move the relevant bits into pkgutil.py and into a new
>eggutil.py.
...
>You can easily provide a pkg_resource.py module with your old API
>that interfaces to the new reorganized code in the stdlib.

...and are you proposing that this other module *also* be included in 
the stdlib?  If so, that was not clear from your previous messages.


> > That's not even remotely similar to "take it or leave it".  It might
> > *seem* that way, of course, simply because in any proposal to change the
> > API, there's an implicit question of why nobody proposed the change via
> > the Distutils-SIG, sometime during the last 2+ years of discussions
> > around that API.
>
>This doesn't have anything to do with distutils. It's entirely
>about the egg distribution format.

Huh?  I'm saying that the pkg_resources API has been being discussed 
on the Distutils-SIG for a good 2 years now.  If there was something 
that users *wanted* to change in the API, surely it would've been 
discussed by now?


>I'm not sure what you want to hear from me.

A good place to start would be the rationale for your proposed API changes.


>You asked for comments, I wrote back and gave you comments.

Yep, and I explained my take on them.  You then brought up this whole 
"take it or leave it" thing, in response to which I attempted to 
provide further clarification of my reasoning -- none of which 
involves any "take it or leave it"-ness, from my perspective.


>  I also
>made it clear why I think that breaking up the addition into different
>PEPs makes a lot of sense and why separating the code into different
>modules for the same reason makes a lot of sense as well.

Actually, no, you didn't.  You simply asserted that certain things 
would be "better" (without any mention of *how* they would be 
better), and that other things "should probably" be done (without any 
explanation of *why*).

So, I simply responded with information of why I did *not* see these 
changes to be useful in any self-evident way, and why I'd want to see 
some justification that would weigh against the PEP's raison d'etre 
-- supporting the existing user base and making it easier for more 
people to join that user base.


>I also tried to stir up some discussion to make life easier
>for setuptools by suggesting a user-package directory on
>sys.path and adding support for namespace packages as
>general Python feature instead of hiding it away in
>pkg_resources.py.

The first is a great discussion topic, but unrelated to the 
PEP.  I'll be happy to participate in that discussion, if I have any 
input to offer.

The second, I don't see as "hiding away"; I would actually suggest 
that pkgutil.extend_path simply be deprecated in favor of the 
pkg_resources API for namespace packages.

I'm not aware of there being a particularly large user base for the 
pkgutil.extend_path, so I don't see a need to change an API that's 
used a lot, to match one that's not used as much.  AFAIK, extend_path 
was originally created for Zope Corp., and also AFAIK, they are now 
using pkg_resources instead.  See also this previous distutils-sig 
discussion (as I said, the API *does* get discussed there):

  http://mail.python.org/pipermail/distutils-sig/2006-August/006608.html

You'll notice that in my response, I also left the door open to 
supporting .pkg files (an extend_path feature not supported by 
pkg_resources), if somebody could tell me what they're used 
for.  Nobody has responded to that, so far.


>You should see this as chance to introduce new concepts to Python.
>Instead you seem to feel offended every time someone suggests a
>change in your design.

I'm not offended.  I've simply commented on your comments, and 
suggested that some of them are off-topic in relation to the PEP, or 
shown why they would be counter to the expressed purposes of the PEP.

As for introducing new concepts to Python, IIRC, Guido and Jim Fulton 
co-invented namespace packages and pkgutil.extend_path() quite a few 
years ago, so I'm not really the introducer there, nor is it a 
particularly new concept.  So I'm not sure what new concepts you're 
talking about.


>That's also the reason why I stopped
>discussing things with you on the distutils list. There was simply
>no way of getting through to you.

We simply have different goals.  In the case of the distutils-sig, my 
perception is that your proposals were aimed at preserving the 
investment of a small number of expert distutils users, at the 
expense of the broader public who knew next to nothing about the 
distutils.  Those proposals "got through to me" just fine; I just 
didn't (and don't) agree with them as goals for setuptools, in any 
place where they conflicted with setuptools' primary goals (which 
included adoption by *new*, distutils-ignorant and/or 
distutils-disliking users).