[Python-Dev] PEP 376 and PEP 302 - allowing import hooks to provide distribution metadata

[Paul Moore]

2009/7/4 Brett Cannon <brett at python.org>:
>> > Finder.uninstall(distribution_name, filter=callable, installer=name)
>> > Uninstall the given distribution. It's likely that many finders will
>> > be read-only. In that case, this function should return None.
>> > Otherwise, return a list of the "files" removed. (This may need some
>> > clarification, as many finders won't have a concept of a "file name").
>>
>> PEP 302 finders and loaders actually *are* expected to have a concept of
>> a "file name". For Python modules they they are expected to have it so
>> they can populate __file__ correctly
>
> Not only that, lots of code out there expects values from __path__ to
> contain os.path.sep on top of __file__, so the idea of file paths is already
> hard-coded into how import works, whether we like it or not.

Yes, consider me duly reprimanded :-) They aren't *file* names, but
they still have to look similar enough to satisfy code that uses
things like filename manipulation functions on them.

>> (runpy is also a lot happier with
>> them when they expose get_filename(), a relatively recent addition to
>> the PEP 302 API).
>
> An addition I was not even aware of. Looks like importlib needs a little
> updating.

Yes, I didn't notice that sneak into PEP 302, either :-)

>> For other resources they're expected to have it so the
>> get_data() API can work correctly.
>>
>> One other thing to consider: it may be desirable to have hooks for
>> meta_path and path_hooks in distutils that are checked *before* the full
>> import hooks in the sys module. The reason I say this is that it would
>> allow someone to override just the metadata retrieval (e.g. to pick up
>> the extra dependency information saved by setuptools and friends)
>> without having to worry about breaking the actual imports from those
>> locations.
>
> So as the guy who plans to try to clean up import, I just want to say that
> implicit finders will eventually go away. Do not start off making them
> implicit, but you can have them installed on sys.meta_path or sys.path_hooks
> by default.

I'm against having distutils-specific hooks. I don't see a genuine
need, and it adds complexity so let's wait for a proper use case
before adding that.

With some suitably ugly hacking, you could probably add a finder to
sys.meta_path which never actually found a module, but which reported
one or more distributions for the distutils metadata APIs. Playing
around with that idea could likely give you any flexibility you needed
(at the hopefully minor cost of an extra dummy meta_path call on each
import).

As far as the currently implicit filesystem path handling is
concerned, I'm aware that the intention is for importlib to move these
into an explicit importer at some point. For my prototype changes to
Tarek's prototype, I'm creating a dummy filesystem finder with just
the new methods needed for the distribution metadata. It will be easy
to move these into a real filesystem finder when one emerges.

Paul.