[Distutils] [Python-Dev] PEP 365 (Adding the pkg_resources module)

[Phillip J. Eby]

At 05:04 PM 3/19/2008 +0000, Paul Moore wrote:
>On 19/03/2008, Phillip J. Eby <pje at telecommunity.com> wrote:
> > It's so that you can get loaders for modules that aren't imported yet
> >  -- and your code would need to handle this case too.
>
>Well, technically, it need not, as I don't *need* to implement the
>exact functionality that pkg_resources does. My (personal) goal is to
>standardise the functionality, not to cater for every possible use
>case.
>
> >  (You could handle it by actually going ahead and importing the module, as
> >  pkg_resources does, but there are other tools using pkgutil to e.g.
> >  inspect as-yet-unimported modules.)
>
>I'm not sure what you mean by that. There are no tools using
>pkgutil.resource_stream, as it doesn't exist yet. So whether it
>imports the module or not cannot be relevant (although it should be
>documented).

You asked why pkgutil.get_loader() was more complex than you'd 
expect.  That's the question I answered, in both of the snippets you 
quoted above.  That is, when I said, "your code would need to handle 
this case", I meant "your code for get_loader()", not "your code for 
resource_foo() methods".

IOW, I was trying to encourage you to use get_loader() as-is rather 
than modifying or replacing it with something "simpler" that would 
break other uses of get_loader() in the stdlib.


> > You don't really need to, because even though they're technically
> >  "undocumented", the intent was for them to be published APIs (apart
> >  from simplegeneric, which is an implementation detail).  The docs
> >  just never got copied to the official docs.
>
>If it's just a matter of copying documentation from the PkgResources
>page, that's no issue. However, only get_importer is documented on the
>PkgResources page.

I meant, the documentation was never copied from the *docstrings* of 
these functions in pkgutil; as you noted, not all of them are direct 
ports from pkg_resources.  I refactored the framework there to 
include support for pyrun and pydoc's use cases as well.

It's true that some additional documentation is needed, too, for 
anyone who wants to implement a PEP 302 loader or importer that wants 
to support enhanced features like listing modules or getting resources.


>On reviewing the pkg_resources APIs, I only see the following that
>look worth porting to pkgutil:
>
>     * resource_exists(package, resource_name)
>     * resource_stream(package, resource_name)
>     * resource_string(package, resource_name)
>     * resource_isdir(package, resource_name)
>     * resource_listdir(package, resource_name)
>
>...
>
>Hmm, these go way beyond the simple (and optional) loader.get_data
>interface from PEP 302. And yet the pkg_resources machinery is far
>more complex than I'd want to see in the stdlib. Maybe I'll just
>reinvent the wheel and see how far I get :-)

So that future PEP 302 importers can define support for these 
features, please use
pkgutil.simplegeneric to implement these, following the pattern set 
out by other pkgutil functions.  E.g.:

     def resource_listdir(package, resource_name):
         return loader_listdir(get_loader(package), package, resource_name)

Where 'loader_listdir' is a simplegeneric with default handlers 
registered for the built-in/stdlib loader types (zipimporter, None, 
and ImpLoader).

If you do this, then new importer/loader types can use 
loader_listdir.register() to support the functionality, without 
monkeypatching or other hacks.