At 12:21 PM 8/19/2005 -0500, Ian Bicking wrote:

When developing a namespace distribution, where should I put it? Should each such distribution be a separate project in the repository (e.g., code goes in Paste/exceptions/trunk/paste/exceptions/)?

If you want to distribute them separately, they need to be separate projects with their own setup.py and project name, like "Paste-Exceptions" or whatever. For example, let's say I made "peak.util" a namespace package. Then, I could create separate projects like "PyUUID", "DynamicImports", "IndentedStream", and so on, for the different modules in the "peak.util" package. Each one would have to have its own, independently distributable project, something like: DynamicImports/ setup.py peak/ __init__.py util/ __init__.py imports.py With the setup.py listing 'peak.util' as a namespace package. The __init__.py files will be empty. The reason you need this layout is that you want "setup.py develop" to still work, and for "develop" and "test" to work, they needs the __init__.py files. If you are only using "install" or you always run from built eggs, you can forego the __init__.py files and even the directory tree, slapping the module directly into the project directory, and setuptools will generate stub __init__.py's for you when it builds the egg. (The livinglogic.de libraries like ll-color do this.) But, if you want to be able to use the "develop" and "test" commands, you need a source layout that includes the package tree. Anyway, the idea behind a namespace package is that you get to claim a package name like "paste", and then put all your modules and packages under it, without forcing somebody to install a single mega-distribution. PEAK right now has a huge array of functionality under "peak.*" packages, and much of it could be used as independent pieces, if only you didn't have to install all of PEAK. Over time (i.e. as soon as I have enough of setuptools finished to have some spare time!), I intend to start breaking out such pieces as separate distributions, but all the code that still imports e.g. "peak.util.imports" will still work, even though it will be in a different distribution. It would be nice, by the way, if somebody who's new to this whole namespace package idea would write up an introduction to this for the setuptools docs, because the idea is so old hat to me that I doubt I can write a really good introduction to it.

Can (or should) I put them all together and let setuptools break them up when I distribute?

Setuptools actually can break them up, using the currently-undocumented "feature" facility. It's undocumented because if you change feature options between builds, you may not get what you expect. But I don't really recommend this approach, because then you have to deal with --with-X/--without-X options, and then the projects would have to have different names, etc. I went through this thought process when thinking about splitting PEAK, as it seemed like it would be easier to leave everything together. But, it's hard to still be compatible with EasyInstall that way. Ultimately I think the "feature" approach is a dead end.

And will I have to move modules out of the namespace package and into subpackages?

No. A namespace package is a package that's a container. The container can be added to by various distributions; each distribution can add modules, resources, or subpackages. The only thing you shouldn't do is run code in the __init__ module, because it will get run as soon as pkg_resources is imported and any distribution for the package is on sys.path. For normal site-packages installation, this means that as soon as pkg_resources is imported, all declared namespace package __init__.py's get run. If you don't want them to get run right away, you can put this boilerplate into all __init__.py files for the package: __import__('pkg_resources').declare_namespace(__name__) and then don't declare the namespace in setup.py. The problem with this approach, however, is that you need to make absolutely certain you do it in the mynamespacepackage/__init__.py file of *each* distribution. If you miss even one, and that missed one gets imported first, then the others will not get added to the namespace. This is why it's preferable to use the documented approach of listing namespace packages in setup(), and always using empty __init__.py files for namespace packages.