[Python-3000] Changing the import machinery

[Phillip J. Eby]

At 04:33 PM 4/20/2006 +0100, "Guido van Rossum" <guido at python.org> wrote:
>On 4/20/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> > Guido van Rossum wrote:
> >
> > > On 4/20/06, in python-dev, Fredrik Lundh <fredrik at pythonware.com> wrote:
> > > > I was hoping that for Python 3.0, we could get around to unkludge the
> > > > sys.path/meta_path/path_hooks/path_importer_cache big ball of hacks,
> > > > possibly by replacing sys.path with something a bit more 
> intelligent than
> > > > a plain list.
> > >
> > > That's an excellent idea. Are there any volunteers here to help out?
> > > Even just listing specific use cases / scenarios that are currently
> > > difficult to solve right would be tremendously helpful. (I think that
> > > Phillip's and others' experience with setuptools might be very
> > > useful.)
> >
> > how far would a plain
> >
> >     the path can contain either directory names or import handlers
> >
> > modification take us ?  (where import handlers are duck-typed objects
> > with an as-small-as-possible interface, and the handler for builtins and
> > frozen objects are included in the list).
>
>Probably all the way. PEP 302's rationale discusses some problems that
>explain why this wasn't done in 2.x; perhaps those concerns can be
>swept in 3.0.

I'm afraid I disagree.  PEP 302 actually has some tremendous advantages 
over a pure objects-on-sys.path approach:

* Strings can be put in any configuration file, and used in .pth files

* Strings can be put in environment variables (like PYTHONPATH).

* Strings can be printed out, with all their contents showing and nothing 
hidden

In short, strings are better for humans.

There are only two things wrong with PEP 302 IMO, and neither is its "fault".

The first is that the "classic" import machinery isn't on sys.meta_path, 
and the 'imp' API isn't defined in terms of PEP 302.  Those two things 
can't change without introducing backward incompatibilities.

The second is that PEP 302 only covers "location" importers, not "format" 
importers.  That is, if you want to do something like make Kid or Cheetah 
templates importable, you have to replace things with new machinery.  This 
is a more important problem to solve, IMO, than coming up with yet another 
set of semantics for the things that *are* covered by PEP 302.

PEP 302 is reasonably clear and concise, although there are occasional 
loopholes.  At least three people have re-implemented parts of it in pure 
Python from the spec: Just van Rossum, Neal Norwitz, and myself (twice, in 
my case).  It is easy and elegant -- the only hard parts are when you have 
to wedge all the "classic" imports in, or if you want to implement "format" 
importers.

So, my vote for Py3K is to move the classic import machinery to explicit 
objects on sys.meta_path -- preferably one for each distinct rule, so that 
it is possible to *remove* them if you don't want them.  (For example, the 
Chandler folks have made a big deal about not wanting general Windows 
registry imports to apply; it would be nice if they could just delete an 
object from sys.meta_path rather than having to build their own Python to 
elide this.)

I would also suggest that we ask Just if he can come up with some elegant 
extension that would allow PEP 302 to support pluggable import *formats* as 
well as locations.  :)