[Python-Dev] making the import machinery explicit

[Brett Cannon]

To start off, what I am about to propose was brought up at the PyCon
language summit and the whole room agreed with what I want to do here, so I
honestly don't expect much of an argument (famous last words).

In the "ancient" import.c days, a lot of import's stuff was hidden deep in
the C code and in no way exposed to the user. But with importlib finishing
PEP 302's phase 2 plans of getting imoprt to be properly refactored to use
importers, path hooks, etc., this need no longer be the case.

So what I propose to do is stop having import have any kind of implicit
machinery. This means sys.meta_path gets a path finder that does the heavy
lifting for import and sys.path_hooks gets a hook which provides a default
finder. As of right now those two pieces of machinery are entirely implicit
in importlib and can't be modified, stopped, etc.

If this happens, what changes? First, more of importlib will get publicly
exposed (e.g. the meta path finder would become public instead of private
like it is along with everything else that is publicly exposed). Second,
import itself technically becomes much simpler since it really then is
about resolving module names, traversing sys.meta_path, and then handling
fromlist w/ everything else coming from how the path finder and path hook
work.

What also changes is that sys.meta_path and sys.path_hooks cannot be
blindly reset w/o blowing out import. I doubt anyone is even touching those
attributes in the common case, and the few that do can easily just stop
wiping out those two lists. If people really care we can do a warning in
3.3 if they are found to be empty and then fall back to old semantics, but
I honestly don't see this being an issue as backwards-compatibility would
just require being more careful of what you delete (which I have been
warning people to do for years now) which is a minor code change which
falls in line with what goes along with any new Python version.

And lastly, sticking None in sys.path_importer_cache would no longer mean
"do the implicit thing" and instead would mean the same as NullImporter
does now (which also means import can put None into sys.path_importer_cache
instead of NullImporter): no finder is available for an entry on sys.path
when None is found. Once again, I don't see anyone explicitly sticking None
into sys.path_importer_cache, and if they are they can easily stick what
will be the newly exposed finder in there instead. The more common case
would be people wiping out all entries of NullImporter so as to have a new
sys.path_hook entry take effect. That code would instead need to clear out
None on top of NullImporter as well (in Python 3.2 and earlier this would
just be a performance loss, not a semantic change). So this too could
change in Python 3.3 as long as people update their code like they do with
any other new version of Python.

In summary, I want no more magic "behind the curtain" for Python 3.3 and
import: sys.meta_path and sys.path_hooks contain what they should and if
they are emptied then imports will fail and None in sys.path_importer_cache
means "no finder" instead of "use magical, implicit stuff".
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120414/d7de5479/attachment.html>