[Python-Dev] Relative Package Imports

[James C. Ahlstrom]

Jim Fulton wrote:
> 
> Gordon McMillan wrote:
> >
> > [Jim Fulton]
> > > Here's an example that I hope will be motivating:
> > >
> > > Suppose Marc-Andre has a package mx with subpackages DateTime
> > > and stringtools.
> > ...
> > > Zope has a notion of products which are *self contained* packages that
> > > are sub-packages of the Products package.  So, suppose someone wants
> > > to write a NiftyDB product, which is a Zope product that provides
> > > access to an external database.  Now the author of the NiftyDB product
> > > wants to use the mx package.  The mx package is not a standard part of
> > > Zope, or of Python, so they simpley include it in the NiftyDB product
> > > directory.

First, I am not all that opposed to having a notion of ".."
available in the import statement.  If we can write zope.dir1.mod1
which is a relative import going down, maybe we can write
../dir2/mod2 or something spelled differently.  But I think
there would still be problems.

We would be relying on all package authors to use ".."
or "__" within their package.  But it is more natural to write
zope.this.that or mx.this.that everywhere, and that is what
people have in fact done.  Probably there would be bugs and
at least annoyance.

And it seems unfortunate to need this feature.  A proper
package scheme should really isolate package internals from
the outside world, and if this isn't happening we need a new scheme.

I also don't think there is any escape from needing standard
package names, at least the head name "zope" for example.

Of course all this currently works if (in Jim's example) both
"zope" and "mx" are top-level names.  Then the ".." is not
necessary.  But, as Jim points out:

>> But why shouldn't it be visible? 

>Because visibility has a cost.  Making mx a top-level name
>means that someone else can't make it a top-level name.
>This is why packages are a good idea.

The global shared nature of PYTHONPATH and its name space makes
it difficult to guarantee that all required packages are going
to be present in a complicated installation like zope.  And if
anyone else installs another Python package, it can easily break
the first installation.

One solution is for an application to establish its own
PYTHONPATH which can not be altered.  If this points to "zopedir"
then the installer can freely install mx to the directory
zopedir/mx and be confident that another mx installation
is not damaged, nor used.

The logical extension is to place each package into its own
file using a scheme like Gordon McMillan is using.  If the
package contents is obtained by seeking from the END of the
file, then multiple package files can be concattenated with
  cat package1 package2 zope mx >> bigpackage
and a large installation like zope can be shipped with its
own "bigpackage" library which is essentially a normal
PYTHONPATH archive with everything above the head directory
names thrown away.  The Python library is included under
"Lib" just as it is now.  The bigpackage library implies
its own PYTHONPATH of "Lib;."  If the Python library files are
placed in the root, the implied PYTHONPATH is ".".

Jim Ahlstrom