[Python-Dev] Re: Proposal for a modified import mechanism.

[eric]

I have to agree with Prabhu on this one.  The current behavior of import,
while fine for standard modules and even simple packages with a single
level, is sub-optimal for packages that contain sub-packages.  The proposed
behavior solves the problem.

Handling the packaging issues in SciPy was difficult, and even resulted in a
(not always popular) decision to build and overwrite the Numeric package on
machines that install SciPy.  Prabhu's import doesn't resolve all the issues
(I think packages may just be difficult...), but it would have solved this
one.  The proposed import allows us to put our own version of Numeric in the
top SciPy directory.  Then all SciPy sub-packages would grab this one
instead of an existing site-packages/Numeric.  That makes SciPy
self-contained and allows people to try it out without worrying that it
might break their current installation.  There are other solutions to this
problem, but Prabhu's fix is by far the easiest and most robust.

Prabhu's import also has some other nice benefits.  Some of the sub-packages
in SciPy are useful outside of SciPy.  Also sometimes it is easier to
develop a packages outside of the SciPy framework.  It would be nice to be
able to develop a module or package 'foo' outside of SciPy and then move it
into SciPy at a later date.  However, every SciPy sub-package that referred
to foo prior to its inclusion in SciPy now has to be updated from 'import
foo' to 'import scipy.foo'.  These kind of issues make it very painful and
time consuming to rearrange package structures or move modules and
sub-packages in and out of the package.  Simplifying this will improves
package development.

> I'm personnally against anything that enlarges the search path uselessly;

Hopefully I've explained why it is useful for complex packages.

> because the obvious reason of increased name space collision, increased
> run-time overhead etc...

I'm missing something here because I don't understand why this increases
name space collision.  If the objection is to the fact that SciPy can have a
version of Numeric in it that masks a Numeric installed in site-packages, I
guess I consider this a feature, not a bug.  Afterall, this is already the
behavior for single level packages, extending it to multi-level packages
seems natural.  If this isn't your objection, please explain.

The current runtime overhead isn't so bad.  Prabhu sent me a few numbers on
the SciPy import (which contains maybe 10-15 nested packages).  I attached
them below -- the overhead is less than 10%.  It should be negligible for
standard modules as only packages are really affected (right Prabhu?).

$ python
>>> import time
>>> s = time.time (); import scipy; print time.time()-s
1.37971198559
>>>
$ python
>>> import my_import
>>> import time
>>> s = time.time (); import scipy; print time.time()-s
1.48667407036

There may be technical issues under the covers that make this hairier than
it appears, but, from the standpoint of someone working on a large
multi-level package, it looks like a good idea.

see ya,
eric

----- Original Message -----
From: "Frederic Giacometti" <frederic.giacometti@arakne.com>
To: <import-sig@python.org>
Cc: <ej@ee.duke.edu>; <prabhu@cyberwaveindia.com>; <python-list@python.org>
Sent: Saturday, November 10, 2001 1:13 PM
Subject: Re: Proposal for a modified import mechanism.


>
>
> >
> > I'd like to ask the Python developers if they'd consider
> >
> >     (a) changing the way the current import works to do what I
> >     proposed, or,
> >
> >     (b) add a new keyword like 'rimport' (or something) that does this
> >     recursive search through parent packages and loads modules.  This
> >     was actually suggested by Gordon McMillan.  Gordon actually
> >     suggested something stronger -- import only supports absolute
> >     names, rimport is relative import and rrimport is a recursive
> >     relative import.  But this would break the current import since
> >     import currently aupports some relative lookup.  So maybe import
> >     and rimport is a workable solution?
>
> I'd rather introduce a __parent__ module attribute (in addition to the
> existing __name__) so that, for instance, the following would do your job:
>
> from __parent__.__parent__.toto import something
>
> In its spirit, this is similar to the '..' of the file systems.
>
> For top-level modules, __parent__ would be None.
>
> I'm personnally against anything that enlarges the search path uselessly;
> because the obvious reason of increased name space collision, increased
> run-time overhead etc...
>
> Frederic Giacometti
>
>
>