[Import-SIG] New PEP Draft: Import Engine

[P.J. Eby]

At 02:05 PM 7/31/2011 +0200, Greg Slodkowicz wrote:
>Necessary changes to finder/loader interfaces:
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>``find_module`` (cls, fullname, path=None, **engine=None**)
>
>``load module`` (cls, fullname, path=None, **engine=None**)
>
>The only difference between 'new style' and PEP 302 compatible
>finders/loaders is the presence of an additional ``engine`` parameter.
>This is intended to specify an ImportEngine instance or subclass there
>of.  This parameter is optional so that the 'new style' finders and
>loaders can be made backwards compatible by falling back on
>engine.sysengine with the following simple pattern:

I see how you can make new style loaders callable from the old 
system, but how do you make *old* loaders usable from the *new* 
system?  That is, I don't see how this proposal is backwards 
compatible with PEP 302.

For that, I think you'd have to define new, optional method names for 
the methods that accepted an engine parameter, with the engine 
falling back to calling the PEP 302 names if the new ones weren't available.


>The existing importlib implementation depends on several functions
>from ``imp``, Python's builtin implementation of ``__import__``
>located in *Python/import.c*. These functions are unaware of
>ImportEngine and place the newly imported module in ``sys.modules``.
>Naturally, this is a problem from the ImportEngine point of view.

It's a general backwards compatibility problem, since importers in 
general are able to assume (and often do) that the loaded modules 
will be placed in sys.modules.


>Similarly, ``imp.NullImporter`` implements a ``load_module`` method
>which is incompatible with 'new style' loaders.

Again, if you use PEP 302 methods only as compatibility fallbacks, 
this won't be an issue.

The biggest problem I see with this as a PEP is that there isn't any 
discussion of backwards compatibility, in the sense that the PEP is 
all about how things *aren't* going to be backwards compatible, and 
the Rationale doesn't present any specific use cases that would 
justify the created incompatibilities.

It would be much better if you can reframe your proposal in terms of 
*additions* to the PEP 302 protocol, rather than *changes*.