[Python-3000] Interest in PEP for callbacks on module import

[Christian Heimes]

Nick Coghlan wrote:
> I'm far from convinced that a from-scratch rewrite (particularly in C) 
> is a great idea myself - that's why I suggested a PEP to look at some of 
> the issues.

Yesterday I wrote down a quick survey of codes with some use cases. Then
I compared my ideas with PJE's import tool. His working implementation
is almost identical to my idea. In my opinion we can copy his import
callback API and re-implement it in C.

> Some curly questions I thought of myself:
> 
> - What do we do if something is inserted in sys.modules directly, and 
> then imported later? Do registered callbacks for that module trigger at 
> all? If so, when?

I'll answer the question with a list of use cases

* a new callback is registered and module is not loaded
  o callback is added to a registry: module name -> list of callbacks

* module is inserted into sys.modules[] manually
  o nothing happens

* module is loaded with __import__() but callbacks weren't fired yet
  o the existing import mechanism is called
  o after the module is loaded and inserted into sys.modules
    the callbacks are fired in reverse order (FILO). The
    callbacks are also called when the module is already in
    sys.modules but the value of the registry isn't None.
  o registry[module name] value is set to None

* module is loaded and registry.get(modulename) returns None
  o nothing happens. Either the callbacks are already fired or
    no callback was registered.

* new callback is registered, module is already loaded
(registry.get(modulename) is None)
  o callback is fired immediately with sys.modules[modulename] as argument

> - Does the lazy importing code need to reacquire the import lock before 
> loading the module on attribute access?

I'm not sure but I'd guess that the lock should be acquired before the
real module is loaded. Does the code need to acquire the lock before the
facade module for lazy importing is created and added to sys.modules?

> Yeah, the more I think about it, the more I think a new PEP for lazy and 
> weak importing is the right way to go (although it would be nice if 
> using the latter didn't automatically cause the former). I'll put an 
> initial draft together (unless someone else beats me to it - I almost 
> certainly won't find time to do it until after Christmas)

What's the difference between lazy and weak importing?

Christian