[Python-Dev] PEP: Post import hooks

[Christian Heimes]

Phillip J. Eby wrote:
[...]

> There's also one twist that I haven't sorted out yet: "Importing" 
> guarantees that a parent module 'foo' will have a 'bar' attribute for 
> the 'foo.bar' module, if 'foo.bar' is lazy.  It does this by 
> registering a callback, ideally *before* any other callback is 
> registered for 'foo' or 'foo.bar' that would look at 'foo.bar'.  I 
> don't see how to maintain this condition in a world where import 
> callbacks can be registered independently.

I've moved the PyImport_NotifyModuleLoaded() call to import_submodule().
It (should) guarantee that the hooks for a parent package is called
before the hooks for its children are called. I've analyzed the code
carefully enough to be sure but all unit test results are on my side.

On other words "import a.b.c" fires the hooks for "a", then "a.b" and at
last "a.b.c".

I could also modify imp.notify_module_loaded to accepts the module name
as string ("a.b.c."). If the module is provided by name (e.g. "a.b.c.")
rather than by object it makes sure that the hooks for "a", "a.b" and
"a.b.c" are called in the right order.

> Bleah.  All of the above isn't really a good explanation of the 
> problem.  Let me try to simplify it:
> 
> * Lazy importing needs to guarantee that foo.bar = 
> sys.modules['foo.bar'], when callbacks for 'foo.bar' execute (in case 
> they refer to foo.bar)
> 
> * To do this, it registers a callback that sets foo.bar = 
> sys.modules['foo.bar'], and does not actually register any foo.bar 
> callbacks until 'foo' is really imported (and thus foo.bar gets set 
> by that callback)

Would the modification fulfill your needs if
imp.notify_module_loaded("foo.bar.baz") call the hooks for "foo",
"foo.bar" and "foo.bar.baz" in that order?

> In the case of the PEP, it's harder for me to figure out what 
> happens, because you might not have any lazy modules around, and the 
> foo.bar issue would then not come up.  You also have the possibility 
> of a problem where a lazy import callback occurs in 3rd party code, 
> while callbacks are occurring from the import machinery.  (Which 
> means that the notification API should probably set the hooks entry 
> to None while it's running, so that if it's called from inside a 
> hook, it will not double-run the hooks, and new hooks registered 
> while hooks are running will get run immediately as they are 
> encountered, instead of getting added to the list.)

The initial design used to set the hooks to None *after* the hooks were
called. I removed code yesterday because I thought it's not required.
Today I've re-added the checks for Py_None. I'm not setting the hooks to
Py_None before the hook are called.

Christian