[Python-3000] Interest in PEP for callbacks on module import

[Christian Heimes]

Phillip J. Eby wrote:
 > Thinking that they can, and actually producing a tested
> implementation of one that does not, in fact, conflict with the 
> implementation of the other, are two different things.
> 
> My concern is that I doubt that you can implement the post-import 
> hook without in the process making it difficult for anyone else to 
> implement a lazy module system that works with it.
> 
> To put it another way, the only way a post-import callback can know 
> that a lazy module has been loaded, is if in some respect the lazy 
> module system *lets* it know.
> 
> Conversely, if a lazy module system puts a module into sys.modules, a 
> callback system that isn't based on module *objects* (as opposed to 
> module names), will incorrectly believe that the module is already 
> imported and fire the callback...  causing the lazy module to be 
> imported, thereby breaking the laziness.
> 
> And this is only the problem that I'm aware of for *my* concept of 
> lazy modules.  There are, I believe, at least two other lazy module 
> libraries out there.
> 
> So, if you propose to separate the mechanism, I think the burden is 
> on the proposer to show *how* this separation can be achieved.

I see your point. The post import hook would indeed conflict with the
laziness of lazy imports if a hook is defined on a module which is
loaded lazy. I wasn't that far in my proposal and I haven't thought
about the implications yet. Thanks for pointing to the problem!

The touch points between post import hooks and lazy imports are minimal.
The lazy module implemenation needs a way to notify the post import hook
that a module was loaded. On the other hand the post import hook just
needs way to detect if a module was loaded for real. Unless I'm totally
mistaken the problem is reduced to two steps in the load mechanism and
to two API methods:

* The notification mechanism of the post import hook must check if a
module was loaded for real before it fires the callbacks.

* The lazy module implementation must notify the post import hook that a
module was loaded.

In Python/C pseudo code:

PyModule_PostImportHooks = {}

def PyModule_IsLoaded(module):
    return getattr(Module, '__lazy__', False)

def PyModule_NotifyPostLoaded(module):
    if not PyModule_IsLoaded(module):
        return
    hooks = PyModule_PostImportHooks.get(module.__name__, None)
    if hooks is None:
        return
    for hook in hooks:
        hook(module)

Lazy modules and post import hooks can still be developed independently
if we agree on names and signature of the two functions. Lazy modules
can implement the PyModule_NotifyPostLoaded() function as NOOP and vice
versa.

Or do you see other connection points?

> The "Importing" package just lets the chain stop, and the remaining 
> callbacks are never called.  My thought is that hooks should be 
> responsible for their own error logging or whatnot.  Letting the 
> failure pass through ensures that the programmers who wrote the hooks 
> clean up their code.  :)

Sounds good to me! :)

Christian