[Python-Dev] PEP 382: Namespace Packages

[glyph at divmod.com]

On 04:56 am, pje at telecommunity.com wrote:
>At 03:58 AM 4/17/2009 +0000, glyph at divmod.com wrote:
>>Just as a use-case: would the Java "com.*" namespace be an example of 
>>a "pure package with no base"?  i.e. lots of projects are in it, but 
>>no project owns it?
>
>Er, I suppose.  I was thinking more of the various 'com.foo' and 
>'org.bar' packages as being the pure namespaces in question.  For 
>Python, a "flat is better than nested" approach seems fine at the 
>moment.

Sure.  I wasn't saying we should go down the domain-names-are-package- 
names road for Python itself, just that "com.*" is a very broad example 
of a multi-"vendor" namespace :).
>>Entries on __path__ slow down import, so my understanding of the 
>>platonic ideal of a system python installation is one which has a 
>>single directory where all packages reside, and a set of metadata off 
>>to the side explaining which files belong to which distributions so 
>>they can be uninstalled by a package manager.

>True... except that part of the function of the PEP is to ensure that 
>if you install those separately-distributed modules to the same 
>directory, it still needs to work as a package and not have any inter- 
>package file conflicts.

Are you just referring to anything other than the problem of multiple 
packages overwriting __init__.py here?  It's phrased in a very general 
way that makes me think maybe there's something else going on.
>>So another clarification I'd like in the PEP is an explanation of 
>>motivation.  For example, it comes as a complete surprise to me that 
>>the expectation of namespace packages was to provide only single- 
>>source namespaces like zope.*, peak.*, twisted.*.  As I mentioned 
>>above, I implicitly thought this was more for com.*, 
>>twisted.plugins.*.
>
>Well, aside from twisted.plugins, I wasn't aware of anybody in Python 
>doing that...  and as I described, I never really interpreted that 
>through the lens of "namespace package" vs. "plugin finding".

There is some overlap.  In particular, in the "vendor distribution" 
case, I would like there to be one nice, declarative Python way to say 
"please put these modules into the same package".  In the past, Debian 
in particular has produced some badly broken Twisted packages in the 
past because there was no standard Python way to say "I have some 
modules here that go into an existing package".  Since every 
distribution has its own funny ideas about what the filesystem should 
look like, this has come up for us in a variety of ways.

I'd like it if we could use the "official" way of declaring a namespace 
package for that.
>>Right now it just says that it's a package which resides in multiple 
>>directories, and it's not made clear why that's a desirable feature.
>
>Good point; perhaps you can suggest some wording on these matters to 
>Martin?

I think the thing I said in my previous message about "multiple 
distributions" is a good start.  That might not be everything, but I 
think it's clearly the biggest motivation.
>>Okay.  So what I'm hearing is that Twisted should happily continue 
>>using our own wacky __path__-calculation logic for twisted.plugins, 
>>but that *twisted* should be a namespace package so that our separate 
>>distributions (TwistedCore, TwistedWeb, TwistedConch, et. al.) can be 
>>installed into separate directories.
>
>Yes.

I'm fairly happy with that, except the aforementioned communication- 
channel-with-packagers feature of namespace packages; they unambiguously 
say "multiple OS packages may contribute modules to this Python 
package".
>Thanks for taking the time to participate in this and add another 
>viewpoint to the mix, not to mention clarifying some areas where the 
>PEP could be clearer.

My pleasure.