[Distutils] [Python-Dev] PEP 365 (Adding the pkg_resources module)

[Phillip J. Eby]

Quick summary of the below: I'm definitely fine with doing a simpler, 
pure-bootstrap module, if there's some consensus on what should go in 
it.  I just wish we could've had this discussion last year, when OSAF 
was still able to fund the work...  ;-)


At 06:13 PM 3/16/2008 -0500, Guido van Rossum wrote:
>Phillip asked me to give an opinion on his pkg_resources PEP. While
>the PEP is short and sweet, the pkg_resources module itself is huge
>(1800 non-blank lines; 16 classes plus 5 exceptions; it exports 67
>names in total according to __all__). And pkg_resources.txt is another
>1700 lines of documentation. I find that hard to swallow. Is there
>anyone besides Phillip who can claim he understands this module?

Bob Ippolito actually wrote the very first version of 
pkg_resources.  Others, such as Philip Jenvey of the Jython project, 
have provided patches.  From previous discussions on the 
distutils-sig, I know that Jim Fulton has in-depth knowledge of both 
pkg_resources and easy_install.

Of course, that's not the same as any of these guys volunteering to 
be maintainers.  :)


>If its inclusion is really meant just as a bootstrap to simplify
>installing other package management solutions, as the PEP claims, I
>would prefer to see something with a much smaller footprint.

Actually, the PEP says:

"pkg_resources is a module used to find and manage Python 
package/version dependencies and access bundled files and resources, 
including those inside of zipped .egg files. Currently, pkg_resources 
is only available through installing the entire setuptools 
distribution, but it does not depend on any other part of setuptools; 
in effect, it comprises the entire runtime support library for Python 
Eggs, and is independently useful."

This kind of glosses over the part where this is also for runtime 
support of projects that use eggs.  Which, these days, is, well, 
almost any large Python project, from Chandler to Enthought to Zope.


>  Surely
>there is no need for example to have support for C extensions inside
>zip files *as part of the bootstrap module*?

It's a runtime; the PEP actually merely proposes that a further 
addition to be made to support bootstrapping, *also*.  Otherwise, the 
PEP would be even shorter.  :)

The reason I proposed it this way was for simplicity -- and politics.

Currently, people using setuptools in their setup.py have to include 
a similar bootstrap module to download setuptools if it's not 
available, and pkg_resources already has version checking logic and 
everything needed to find dependencies and download them.  (Plus, I 
figured it'd be easier to just use what was already there and stable, 
rather than creating something different.)

That was the simplicity part.  The politics part was that:

1. I thought it would be less controversial to include the "runtime 
for eggs" than to include something that's just a bootstrapper for 
setuptools.  However, MvL surprised me by actually being in *favor* 
of including a setuptools bootstrapper.

2. I thought that it would have broader acceptance if it was oriented 
towards bootstrapping *any* package, not just setuptools.

So, if the consensus is that it would be better to have a module that 
only does bootstrap installs of pure-Python eggs from PyPI, I'm 
totally fine with that.


>Unless I find someone besides Phillip who is interested in having this
>included and is willing to help maintain it, I don't really think it
>would be wise to accept this into the standard library.
>
>Phillip, in the PEP you mention that there are several other package
>management tools that also like to use pkg_resources. Maybe you can
>get some folks from those tools to speak up and explain what
>pkg_resources means to them, and maybe even volunteer to co-own it
>once it's in the standard library?

The distutils-sig is the de facto place for discussions regarding 
those tools, so I've cc'd this there.  Hopefully, one or more 
volunteers will step up if they want this.