[Python-Dev] making the import machinery explicit

[Brett Cannon]

On Sat, Apr 14, 2012 at 17:12, Eric Snow <ericsnowcurrently at gmail.com>wrote:

> On Sat, Apr 14, 2012 at 2:03 PM, Brett Cannon <brett at python.org> wrote:
> > To start off, what I am about to propose was brought up at the PyCon
> > language summit and the whole room agreed with what I want to do here,
> so I
> > honestly don't expect much of an argument (famous last words).
> >
> > In the "ancient" import.c days, a lot of import's stuff was hidden deep
> in
> > the C code and in no way exposed to the user. But with importlib
> finishing
> > PEP 302's phase 2 plans of getting imoprt to be properly refactored to
> use
> > importers, path hooks, etc., this need no longer be the case.
> >
> > So what I propose to do is stop having import have any kind of implicit
> > machinery. This means sys.meta_path gets a path finder that does the
> heavy
> > lifting for import and sys.path_hooks gets a hook which provides a
> default
> > finder. As of right now those two pieces of machinery are entirely
> implicit
> > in importlib and can't be modified, stopped, etc.
> >
> > If this happens, what changes? First, more of importlib will get publicly
> > exposed (e.g. the meta path finder would become public instead of private
> > like it is along with everything else that is publicly exposed). Second,
> > import itself technically becomes much simpler since it really then is
> about
> > resolving module names, traversing sys.meta_path, and then handling
> fromlist
> > w/ everything else coming from how the path finder and path hook work.
> >
> > What also changes is that sys.meta_path and sys.path_hooks cannot be
> blindly
> > reset w/o blowing out import. I doubt anyone is even touching those
> > attributes in the common case, and the few that do can easily just stop
> > wiping out those two lists. If people really care we can do a warning in
> 3.3
> > if they are found to be empty and then fall back to old semantics, but I
> > honestly don't see this being an issue as backwards-compatibility would
> just
> > require being more careful of what you delete (which I have been warning
> > people to do for years now) which is a minor code change which falls in
> line
> > with what goes along with any new Python version.
> >
> > And lastly, sticking None in sys.path_importer_cache would no longer mean
> > "do the implicit thing" and instead would mean the same as NullImporter
> does
> > now (which also means import can put None into sys.path_importer_cache
> > instead of NullImporter): no finder is available for an entry on sys.path
> > when None is found. Once again, I don't see anyone explicitly sticking
> None
> > into sys.path_importer_cache, and if they are they can easily stick what
> > will be the newly exposed finder in there instead. The more common case
> > would be people wiping out all entries of NullImporter so as to have a
> new
> > sys.path_hook entry take effect. That code would instead need to clear
> out
> > None on top of NullImporter as well (in Python 3.2 and earlier this would
> > just be a performance loss, not a semantic change). So this too could
> change
> > in Python 3.3 as long as people update their code like they do with any
> > other new version of Python.
> >
> > In summary, I want no more magic "behind the curtain" for Python 3.3 and
> > import: sys.meta_path and sys.path_hooks contain what they should and if
> > they are emptied then imports will fail and None in
> sys.path_importer_cache
> > means "no finder" instead of "use magical, implicit stuff".
>
> This is great, Brett.  About sys.meta_path and sys.path_hooks, I see
> only one potential backwards-compatibility problem.
>
> Those implicit hooks were fallbacks, effectively always at the end of
> the list, no matter how you manipulated the them.  Code that appended
> onto those lists would now have to insert the importers/finders in the
> right way.  Otherwise the default hooks would be tried first, which
> has a good chance of being the wrong thing.
>
> That concern aside, I'm a big +1 on your proposal.


Once again, it's just code that needs updating to run on Python 3.3 so I
don't view it as a concern. Going from list.append() to list.insert() (even
if its ``list.insert(hook, len(list)-2)``) is not exactly difficult.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20120414/ae5e296a/attachment.html>