[Python-Dev] Write All New Import Hooks (PEP 302) in Python, Not C

[Just van Rossum]

Samuele Pedroni wrote:

> I agree with Guido that special cookies can be used for normal
> importers.

I find this rather ugly, and would only do that if it's crucial that the
pathless importer should be invoked somewhere in the middle of sys.path.
And I don't have a use case for that. There are plenty of use cases for
sys.meta_path.

> I see also the point for such really _meta_ importers. But if one has
> many of them how the code that installs them respectively the user
> decide on the order of invocation.

I have no idea what you mean by this.

> It seems that theoretically to let code programatically alter
> meta_path, there's a need for categories/priorities for meta
> importers:
> 
> - loggging importers come before everything - global importers after
> etc
> 
> Lacking this, it seems equivalent to installing __import__ hooks
> apart a tiny bit of convenience.

I disagree that it's only a "tiny bit" of convenience:

- PEP-302-style importers don't have to deal with the "is this module
already loaded" issue, it will only be invoked for new imports.

- They don't have to deal with complete dotted name imports, they only
get invoked for the head of a dotted name. The rules for dotted name
imports are very subtle and are all taken care of by the surrounding
code, so this really is a big win.

> > There should be only one way to do it if import.c did it only one
> > way. Overriding or emulating builtin and frozen imports *must* be
> > possible, and sys.meta_path is the feature for it. A key aspect of
> > sys.meta_path is that it allows to refactor import.c in a really
> > nice way; in the future, sys.meta_path would look like this:
> >
> > [BuiltinImporter, FrozenImporter, PathImporter]
> >
> > sys.meta_path is then the central import dispatcher. PathImporter
> > would be responsible for sys.path/pkg.__path__ handling and
> > invoking sys.path_hooks and dealing with sys.path_import_cache. It
> > is *key* to PEP 302, and to me simply a killer feature.
> 
> This would be nice, but if the above point is not resolved

Which point?

> and this
> is not concretely implemented, we risk to have meta_path used in
> non-intended ways or be as much problematic as __import__.

The new import hooks operate at *such* a different level that __import__
that I honestly don't see what the problem is.

> Lacking precise policies and clear-cut commitment for its future
> uses, it's better left out (it can be added in the future).

I disagree. Better be able to *use* it and establish conventions based
on actual usage than leave it out and fantasize about what it *could*
do.

PEP 302 should replace 99% of the use cases from __import__ and
hard-coded hooks in import.c. Leaving out sys.meta_path would reduce
this number significantly and would be a really dumb decision.

Just