[Python-Dev] New universal import mechanism ( Re: [Python-checkins] python/dist/src/Python import.c,2.210,2.211)

[Just van Rossum]

Greg Ward wrote:

> On 02 December 2002, Just van Rossum said:
> > I've been doing some thinking about the import mechanism, too, and my
> > current gripe is about the __import__ hook: I find it utterly useless
> > that the implementer of the hook is supposed to deal with
> > sys.modules. What I wish for is a hook that only gets invoked if the
> > module isn't already in sys.modules.
> 
> I can see where you're coming from, but one disadvantage to your scheme
> is that it would not be possible to override .py/.pyc/.pyo files -- they
> would always be imported in preference to any other scheme.  This could
> well be a feature.  ;-)

I don't follow. In my scheme (which in the meantime I've partially implemented)
any first import will be done by an import hook on sys.import_hooks, so you have
full control. If you mean that you can't hook imports that are already in
sys.modules: yes, that's a feature ;-). On the other hand, the existing
__import__ hook will not change, so you can still do it using __import__. Look
at it this way: the default __import__ implements the new scheme; it's the low
level hook. For 99.9% of import hook needs you wouldn't touch it, though. Here's
the protocol that my current implementation uses:

    - sys.import_hooks is a list of hooks
    - a hook is a callable object taking two arguments:
      [sub]modulename and path. Path is either the parents package's
      __path__ or None.
    - a hook either returns None ("module not found") or a two-tuple:
      (loadfunc, cookie). loadfunc is a callable object that will
      be called with two arguments: the fullname and "cookie". The
      cookie is just an arbitrary object, private to the hook. The
      loaderfunc must return the imported module.
    - (Currently it expects that the loaderfunc will insert the module
      in sys.path, I'm not sure I like that. Actually, I am sure I
      don't like that <wink>, but changing that would require more
      changes to import.c that I'd like. I'll have to investigate.)

It plugs neatly into import_submodule(), right now it's a tiny patch.

Just