[Python-Dev] Relative Package Imports

M.-A. Lemburg mal@lemburg.com
Fri, 17 Sep 1999 11:33:43 +0200

Guido van Rossum wrote:
> > I'm wasn't planning to write my own hook until now. I need this
> > feature to be able to organize my package internal stuff swiftly
> > and easily, plus to be able to possibly put it under new top-level
> > packages. Obviously I seem to be more or less alone with this need, so
> > I'll stop argueing for inclusion of "something like relative imports"
> > in the distribution.
> I still hope against hoping to convince you not to do this.  I think
> this adds lots of complexity for the reader of your package.  I think
> of *all* software that I write as something that eventually someone is
> going to read and try to understand.  I also think of it as something
> that eventually someone is going to port to a platform that wasn't
> even designed when I wrote the software.  Tricks like depending on a
> custom import hook make reading a pain.
> For example, the reader who is just starting to browse a large package
> often needs to follow the thread of execution from one module to the
> next.  If the import statements contain magic like __magic__.foo.bar
> (actually proposed magic renamed for generality :-), this reader will
> first need to understand the custom importer -- which is likely one of
> the hariest components of the package.

While it will perhaps look funny to him at first sight, I'm sure
that a proper name will make the idea clear to him:

from __superpackage__.foo import bar

or just

from __super__.foo import bar

[Super Foo -- sounds like a comic star ... there's even integrated
fun in these names ;-)]

> The same thing applies with even more force to tools like package
> browsers.  IDLE for example has a class browser which displays the
> true module and class name for base classes.  E.g. if you have code
> that says
>     from Tkinter import Widget
>     class MyWidget(Widget): ...
> it correctly deduces that MyWidget's base class is Tkinter.Widget.
> (And in a future version it will probably allow you to click through
> to that class, show you its methods, etc.)
> Custom importers break this feature, and thus make the code harder to
> analyze for the reader.

That's why I tried argue for making it a standard: all tools will
then know about the syntax used and take appropriate action.
> (Not all custom importers are bad; there are plenty of reasons to
> augment the standard importer.  But, while custom importers make
> different interpretations of the namespace possible, I frown upon that
> particular use.)
> > (b) is (if at all) a problem only to be taken into account by
> > the author of package X. He may or may not use rel. imports.
> > A 12-year old probably won't (but then: you never know ;).
> Which severely limits your ability to do what you want with packages
> you didn't write.
> I know that this sounds politically incorrect in a radical free
> software world, but often in the end I find it more convenient to
> conform to the rest of the world and "fit in" than to be different in
> every little way.  Note that my hair isn't blue :-); I've also
> replaced my "signature" glasses with a more conventional model.

Naa, I don't want to be different just to be plain silly different.

The case here is one of maintenance. I have around 12 Python packages
which I maintain, some of them public, others proprietary. Due
to the large number I try to make them all as easy to handle as possible.
Now I can't just go and rename the imports in case I want to put
them under some other common package name for application A and
then do the same with another package name for application B. I would
have to do the same thing for every new release of the packages since
there are also others who use the standard mx-based name. I just
want the packages to work whereever I put them, e.g. I could
link them into the new location using symlinks and be done
with the work.
Of course there are other ways to make this work (and I want
to thank everybody who contributed to the sys.path ideas), but
they just add omplexity to a whole different area.

> > BTW, what is this CP4E thing you're talking about. If it's an
> > syntax aware editor, I have a friend who is really interested
> > in these things... could you send me an URL that I can send him ?
> Where have you been?  It's on the python.org homepage, has been
> discussed in c.l.py, c.l.tcl, even c.l.ada, in lwn, and on /.!  The
> syntax aware editor (a proposed super-version of IDLE) is only a small
> part of it.  See python.org/doc/essays/cp4e.html

Nope, haven't heard anything about it -- perhaps I was on
vacation. Sound interesting though. An editor where you can
just type away (and the editor fills in the gaps) would be
great fun.
> > Oh well... I guess they'll have to use 'mex' if they decide to go
> > the reverse domain way ;-)
> Or, in a worse-case scenario, the first Mexican developers using the
> reverse domain will probably not be aware of the mx toplevel package,
> and by the time their software hits the street it will be too late.

Well, I guess then they'll have a good reason to upgrade their
software. Just like all those companies who made extra money
by sticking a "Y2000 + Euro-capable" button on their software.
No, seriously, with the proposed extension (see below) this
shouldn't be much of a problem because my packages all use
capital first letters. The reverse domain name would use lower
case ones, e.g. mx.pythonware.pilios or mx.zopistas...
> > True, perhaps we should lighten this requirement a little when we
> > recode the import mechanism in Python ? E.g. if a local import
> > fails continue the search with the fully qualified name and only
> > if that fails, restart using the local name. This would need some
> > kind of fastpath cache to make the search reasonably fast though.
> Now this is something that I would consider.

Should be easy to add to the imputil based core emulation.
> > Wouldn't it suffice to just put them into one package, e.g.
> > 'python.' ?
> And somehow make all user code live implicitly inside that package?  I
> don't see how that solves anything.

No, to free the top-level namespace and make the origin of the
imports clearer, e.g.

import python.image
import PIL.image
import zope.image


> And believe me, at some point *someone* is going to
> need to rip out your custom importer because it interferes with *his*
> custom importer, and he'll choose to replace all your __ imports with
> absolute package names, rather than trying to add the __ feature to
> his importer.

Note that with imputil doing its thing, he won't have to
worry about adding code to his importer. It'll just pass on the
import to my importer. That's the fun part about imputil.

Marc-Andre Lemburg
Y2000:                                                   105 days left
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/