[Python-Dev] PEP: Post import hooks

[Phillip J. Eby]

At 09:40 PM 1/10/2008 +0100, Christian Heimes wrote:
>Phillip J. Eby wrote:
>[...]
>
> > There's also one twist that I haven't sorted out yet: "Importing"
> > guarantees that a parent module 'foo' will have a 'bar' attribute for
> > the 'foo.bar' module, if 'foo.bar' is lazy.  It does this by
> > registering a callback, ideally *before* any other callback is
> > registered for 'foo' or 'foo.bar' that would look at 'foo.bar'.  I
> > don't see how to maintain this condition in a world where import
> > callbacks can be registered independently.
>
>I've moved the PyImport_NotifyModuleLoaded() call to import_submodule().
>It (should) guarantee that the hooks for a parent package is called
>before the hooks for its children are called. I've analyzed the code
>carefully enough to be sure but all unit test results are on my side.
>
>On other words "import a.b.c" fires the hooks for "a", then "a.b" and at
>last "a.b.c".

Yes, that's the general idea.  But what happens if you are in the 
middle of firing hooks for 'a', and a new hook for 'a.b.c' is 
added?  What about a new hook for 'a'?


>I could also modify imp.notify_module_loaded to accepts the module name
>as string ("a.b.c."). If the module is provided by name (e.g. "a.b.c.")
>rather than by object it makes sure that the hooks for "a", "a.b" and
>"a.b.c" are called in the right order.

Well, it certainly can (and should) do the same if a module object is 
provided, since the module has a __name__.


>Would the modification fulfill your needs if
>imp.notify_module_loaded("foo.bar.baz") call the hooks for "foo",
>"foo.bar" and "foo.bar.baz" in that order?

Only if you can guarantee that no hook for a submodule is run until 
all the parent hooks are finished being called *and* that adding new 
callbacks while callbacks are being run will still call them... 
*after* any already-added callbacks.


>The initial design used to set the hooks to None *after* the hooks were
>called. I removed code yesterday because I thought it's not required.
>Today I've re-added the checks for Py_None.

In general, if you think something in peak.util.imports isn't 
required, you're probably wrong.  ;)


>I'm not setting the hooks to Py_None before the hook are called.

That's fine, but here's a different catch: are you iterating over the 
hooks by taking the length before you start?  If so, then hooks that 
are added *while* the hooks are being called back, will never get 
called, because they'll be added to the end of the list (and you'll 
never reach the end).  Make sure there's a test for that case.

peak.util.imports sets to None after callbacks, but it uses regular 
list iteration, so hooks can be added to the end of the list while 
the hooks are still being called.

An error while running the hooks should also set the hook list to 
None and discard all the hooks.  There isn't any sane way to recover 
from an error in a post-import hook.