[Python-Dev] PEP 0365: Adding the pkg_resources module

[Phillip J. Eby]

At 06:28 PM 5/21/2007 +0200, M.-A. Lemburg wrote:
>However, since this is not egg-specific it should probably be
>moved to pkgutil and get a separate PEP with detailed documentation
>(the link you provided doesn't really explain the concepts, reading
>the code helped a bit).

That doesn't really make sense in the context of the current PEP, 
though, which isn't to provide a general-purpose namespace package 
API; it's specifically about adding an existing piece of code to the 
stdlib, with its API intact.


>What I don't understand about your approach is why importers
>would have to register with the namespace implementation.
>
>This doesn't seem necessary, since the package __path__ attribute
>already provides all functionality needed for redirecting
>lookups to different paths.

The registration is so that when new paths are *added* to sys.path at 
runtime (e.g. by activating a plugin), the necessary __path__ lists 
are automatically updated.  Similarly, when new namespace packages 
are declared, they need their __path__ updated for everything that's 
currently on sys.path.

Finally, there is no requirement that PEP 302 importer strings use 
filesystem-path syntax; a handler has to be registered so that the 
necessary string transformation can be done according to that 
particular importer's string format.  It just happens that zipfiles 
and regular files have a common syntax.  But a URL-based importer, 
LDAP-DN based importer, SQL importer, or other exotica might require 
entirely different string transformations.  All that PEP 302 
guarantees is that sys.path and __path__ lists contain strings, not 
what the format of those strings is.


>Could you add a section that explains the side effects of
>importing pkg_resources ?

The short answer is, there are no side effects, unless 
__main__.__requires__ exists.  If that variable exists, pkg_resources 
attempts to adjust sys.path to contain the requested package 
versions, even if it requires re-ordering the current sys.path 
contents to prevent conflicting versions from being imported.


>The documentation of the module doesn't mention any, but the
>code suggests that you are installing (some form of) import
>hooks.

There are no import hooks, just a registry for registering handlers 
to support other importer types (as seen at the doc link I gave previously).


>Some other comments:
>
>* Wouldn't it be better to factor out all the meta-data access
>   code that's not related to eggs into pkgutil ?!
>
>* How about then renaming the remaining module to egglib ?!

These changes in particular would negate the primary purpose of the 
PEP: to provide a migration path for existing packages using the 
pkg_resources API, including setuptools, workingenv, zc.buildout, etc.


>* The get_*_platform() should probably use the platform module
>   which is a lot more flexible than distutils' get_platform()
>   (which should probably use the platform module as well in the
>   long run)

Please feel free to propose specific improvements to the 
distutils-sig.  But keep in mind that the platform information is 
specifically for supporting .egg filename platform tags.  Where the 
information comes from is less relevant than defining a framework for 
determining compatibility.

I first tried to get some kind of traction on this issue in 2004:

http://mail.python.org/pipermail/distutils-sig/2004-December/004355.html

And so far, the only platform for which something better has emerged 
is Mac OS X, due largely to Bob Ippolito stepping up and submitting some code.


>* The module needs some reorganization: imports, globals and constants
>   at the top, maybe a few comments delimiting the various sections,

I'm not sure I follow you.  Globals such as registries are usually 
defined close to the functions that provide the API for manipulating 
them.  And the rest of the globals (such as working_set), can't be 
defined until the class they're implemented with has been defined.