Python packages - problems, pitfalls.

[Prabhu Ramachandran]

hi,

Recently I discovered that there is some very unfortunate behaviour
with python packages.

Lets say I have a directory 'pkg' that is the root of a package.
Inside this directory I have a sub-package, called sub.  So its
something like:

pkg/
    __init__.py
    a.py
    sub/
        __init__.py
        b.py

Now, from b I'd expect to be able to import 'a' straight away.

Say in b.py I do

import a

This will not work!  I know why it happens but shouldn't Python be
smart enough to avoid such problems?  Why I consider this a major
problem is that the importing of a sub-package depends on where its
parent package is!!  So if pkg is no longer the root of the package
and say its put inside another BigPkg then to make pkg's sub packages
work you'd have to edit *all* the sub packages and change every
reference to pkg.a to BigPkg.pkg.a.  This is a huge pain.

One way around this is to do something like this inside sub's
__init__.py:

import os
__path__.append(os.path.dirname(__path__[0]))

This however will cause the module a to be loaded several times!  Once
as pkg.a and once as pkg.sub.a!!  Imagine what would happen if you had
many such sub-packages.

One way around this would be for import to see if the importee (the
imported package)

  (1) is in the same directory (no problem, this already works)

  (2) is part of a the same package that the importer (the code that
  does the import) belongs to.  If so look in the parent packages
  directory first.

  (3) Keep doing (2) untill you are out of all parent packages.
  Something like walking a tree.

  (4) If all this fails, search for modules in sys.path.

I did some prelimiary searching and came upon the import-sig which led
me on to imputil.  I also found that knee.py does something to
eliminate such problems.  However, I find that

  (1) imputil isn't documented enough (at all?) and I can't really
  find neat examples that illustrate its usage.

  (2) I don't know if using imputil is recommended and what the
  problems are with using it etc.

  (3) knee.py warns me saying """ This code is intended to be read,
  not executed.  However, it does work -- all you need to do to enable
  it is "import knee"."""

I know that I can experiment, read the source, grok it all and look
for solutions but at the moment don't have much time for it.  I'd like
to know what you folks think.  

Are there better ways to get around this packaging problem?  Is this a
know problem that folks are working on??  Why is it that Python does
not deal with this issue more sensibly?


Thanks!

prabhu
-- 
Prabhu Ramachandran			  MayaVi Data Visualizer
http://www.aero.iitm.ernet.in/~prabhu     http://mayavi.sf.net

Where there's no emotion, there's no motive for violence.
		-- Spock, "Dagger of the Mind", stardate 2715.1