[Python-Dev] PEP: Post import hooks

[Christian Heimes]

Phillip J. Eby wrote:
> At 10:14 PM 1/15/2008 +0100, Christian Heimes wrote:
>> My code queues up new hooks while a sequence of hooks is processed. It
>> makes sure that hooks for a parent aren't called in the middle of a
>> child's hook chain.
> 
> Notice that that's not necessary with the notification algorithm I gave,
> since the list in post_import_hooks suffices as a queue.  So, just as in
> peak.util.imports, the registration code doesn't need to know whether
> callbacks are being run; it only needs to know whether they're *finished*.

Are you sure your proposed algorithm and output match for the test case?
I'm confident I got it right in C but I'm getting a different output.

Without the extra imp.notify_module_loaded('a.b') in func_a1(mod)::

   ['func_a1', 'func_a2', 'func_ab1', 'func_ab2', 'func_ab3']


With the extra imp.notify_module_loaded('a.b') in func_a1(mod)::

   ['func_a1', 'func_ab1', 'func_ab2', 'func_ab3', 'func_a2']

I can't see how your implementation results in the first output when
func_a1() calls the notification method.


> Of course, both the notification and registration functions must hold
> the import lock to prevent a race condition where one thread adds a hook
> to the list after another thread has just finished iterating over it and
> is about to replace the list with None.  At least, they have to if
> they're executing any Python code that might cause the GIL to be
> released.  The callbacks will release the GIL, of course, but the
> registration code probably doesn't...  well, it will if it calls the
> hook, and ISTM that the hooks should always execute with the import lock
> held, even if they're fired at registration.

I'm aware of the implications and my code already uses the lock. The
PyImport_NotifyLoaded() method excepts to be called with the importer
lock acquired. So I'm locking the importer lock in
imp_notify_module_loaded(). The PyImport_RegisterPostImportHook() method
does the locking before it accesses sys.modules and sys.post_import_hooks.

Christian