[Python-Dev] Relative Package Imports

[Guido van Rossum]

> I'm wasn't planning to write my own hook until now. I need this
> feature to be able to organize my package internal stuff swiftly
> and easily, plus to be able to possibly put it under new top-level
> packages. Obviously I seem to be more or less alone with this need, so
> I'll stop argueing for inclusion of "something like relative imports"
> in the distribution.

I still hope against hoping to convince you not to do this.  I think
this adds lots of complexity for the reader of your package.  I think
of *all* software that I write as something that eventually someone is 
going to read and try to understand.  I also think of it as something
that eventually someone is going to port to a platform that wasn't
even designed when I wrote the software.  Tricks like depending on a
custom import hook make reading a pain.

For example, the reader who is just starting to browse a large package
often needs to follow the thread of execution from one module to the
next.  If the import statements contain magic like __magic__.foo.bar
(actually proposed magic renamed for generality :-), this reader will
first need to understand the custom importer -- which is likely one of
the hariest components of the package.

The same thing applies with even more force to tools like package
browsers.  IDLE for example has a class browser which displays the
true module and class name for base classes.  E.g. if you have code
that says

    from Tkinter import Widget

    class MyWidget(Widget): ...

it correctly deduces that MyWidget's base class is Tkinter.Widget.
(And in a future version it will probably allow you to click through
to that class, show you its methods, etc.)

Custom importers break this feature, and thus make the code harder to
analyze for the reader.

(Not all custom importers are bad; there are plenty of reasons to
augment the standard importer.  But, while custom importers make
different interpretations of the namespace possible, I frown upon that 
particular use.)

> (b) is (if at all) a problem only to be taken into account by
> the author of package X. He may or may not use rel. imports.
> A 12-year old probably won't (but then: you never know ;).

Which severely limits your ability to do what you want with packages
you didn't write.

I know that this sounds politically incorrect in a radical free
software world, but often in the end I find it more convenient to
conform to the rest of the world and "fit in" than to be different in
every little way.  Note that my hair isn't blue :-); I've also
replaced my "signature" glasses with a more conventional model.

> BTW, what is this CP4E thing you're talking about. If it's an
> syntax aware editor, I have a friend who is really interested
> in these things... could you send me an URL that I can send him ?

Where have you been?  It's on the python.org homepage, has been
discussed in c.l.py, c.l.tcl, even c.l.ada, in lwn, and on /.!  The
syntax aware editor (a proposed super-version of IDLE) is only a small
part of it.  See python.org/doc/essays/cp4e.html

> Oh well... I guess they'll have to use 'mex' if they decide to go
> the reverse domain way ;-)

Or, in a worse-case scenario, the first Mexican developers using the
reverse domain will probably not be aware of the mx toplevel package,
and by the time their software hits the street it will be too late.

> True, perhaps we should lighten this requirement a little when we
> recode the import mechanism in Python ? E.g. if a local import
> fails continue the search with the fully qualified name and only
> if that fails, restart using the local name. This would need some
> kind of fastpath cache to make the search reasonably fast though.

Now this is something that I would consider.

> Wouldn't it suffice to just put them into one package, e.g.
> 'python.' ?

And somehow make all user code live implicitly inside that package?  I 
don't see how that solves anything.

> BTW, as Tim argued: the breakage an easily be leveraged by using
> a smart editor... ;-)

What breakage?  I think __ is a bit too hard to fix easily with a
smart editor.  And believe me, at some point *someone* is going to
need to rip out your custom importer because it interferes with *his*
custom importer, and he'll choose to replace all your __ imports with
absolute package names, rather than trying to add the __ feature to
his importer.

--Guido van Rossum (home page: http://www.python.org/~guido/)