[Distutils] Experimental non-empty namespace package support
Phillip J. Eby
pje at telecommunity.com
Sun Aug 14 23:35:51 CEST 2005
I've just checked in experimental support for lazy, non-empty namespace
packages. And what is that?, you might well ask.
Well, it seems that livinglogic.de distributes certain Python packages
using a distutils kludge that allows a kind of crude namespace package to
exist, without using pkg_resources. Specifically, their 'll-core' project
distributes the 'll' package, and various other projects such as 'll-color'
distribute modules for the 'll' package, but *without including an
__init__.py*. This allows the distutils to install the modules without
overwriting or duplicating the single __init__.py file.
This is an interesting approach to addressing namespace packages in a
pre-setuptools world, but until now it hasn't been really usable with
setuptools, for several reasons:
* pkg_resources only loaded one __init__ file for a given package
* pkg_resources couldn't import modules from a zipfile or directory with
* pkg_resources automatically imports registered namespace packages as
soon as they're discovered, which is not a big deal for empty namespace
packages, but could easily become quite problematic for ones like 'll' that
contain actual code.
So, here's the solution:
* pkg_resources now loads all __init__.py's from all the distributions
that provide contents for the module.
* the bdist_egg command automatically generates a dummy __init__.py for
packages that don't have them. The __init__.py file contains the single line:
This ensures that the other package contents are importable, and that
as soon as an import of the package occurs, it gets registered as a
* projects that use this approach to namespace packaging must *NOT*
pass a 'namespace_packages' argument to setup(), because that would cause
the package to be imported even when it isn't necessary. They must also
put a 'declare_namespace()' call in all of their __init__.py's, to ensure
that the package will become a namespace no matter what the order of
distributions on sys.path is.
Because of the complexity of this approach, I do not recommend it for new
projects. Namespace packages should not have code in any of their
__init__.py files, as it eliminates all of these headaches. So, this is an
experimental feature intended to facilitate backward compatibility
only. Don't expect it to show up in the documentation any time soon!
Anyway, I was able to get this to work for ll-core and ll-color, as long as
I added a 'declare_namespace()' call like the one shown above to ll-core's
__init__.py, and made them both use setuptools. I would appreciate any
feedback that folks have on this.
By the way, because SourceForge CVS updating is slow, it may be a few hours
before you can check out the version of setuptools that supports
this. Make sure you have pkg_resources.py revision 1.69 or higher, and
setuptools/command/bdist_egg.py revision 1.28 or higher, if you want to
More information about the Distutils-SIG