[Python-Dev] PEP 382: Namespace Packages

[Tarek Ziadé]

On Wed, Apr 15, 2009 at 9:22 PM, P.J. Eby <pje at telecommunity.com> wrote:
> At 02:52 PM 4/15/2009 -0400, A.M. Kuchling wrote:
>>
>> On Wed, Apr 15, 2009 at 01:59:34PM -0400, P.J. Eby wrote:
>> > Please see the large number of Zope and PEAK distributions on PyPI as
>> > minimal examples that disprove this being the common use case.  I expect
>> > you will find a fair number of others, as well.
>>   ...
>> > In other words, the "base package" scenario is the exception these days,
>> > not the rule.  I actually know specifically of only one other such
>> > package besides your mx.* case, the logilab ll.* package.
>>
>> Isn't that pretty even, then?  zope.* and PEAK are two examples of one
>> approach; and mx.* and ll.* are two examples that use the base package
>> approach.  Neither approach seems to be the more common one, and both
>> are pretty rare.
>
> If you view the package listings on PyPI, you'll see that the "pure"
> namespaces currently in use include:
>
> alchemist.*
> amplecode.*
> atomisator.*
> bda.*
> benri.*
> beyondskins.*
> bliptv.*
> bopen.*
> borg.*
> bud.*
> ...
>
> This is just going down to the 'b's, looking only at packages whose PyPI
> project name reflects a nested package name, and only including those with
> entries that:
>
> 1. use setuptools,
> 2. declare one or more namespace packages, and
> 3. do not depend on some sort of "base" or "core" package.
>
> Technically, setuptools doesn't support base packages anyway, but if the
> organization appeared to be based on a "core+plugins/addons" model (as
> opposed to "collection of packages grouped in a namespace") I didn't include
> it in the list above -- i.e., I'm bending over backwards to be fair in the
> count.
>
> If somebody wants to do a formal count of base vs. pure, it might provide
> interesting stats.  I initially only mentioned Zope and PEAK because I have
> direct knowledge of the developers' intent regarding their namespace
> packages.
>
> However, now that I've actually looked at a tiny sample of PyPI, it's clear
> that the actual field use of pure namespace packages has positively exploded
> since setuptools made it practical to use them.
>
> It's unclear, however, who is using base packages besides mx.* and ll.*,
> although I'd guess from the PyPI listings that perhaps Django is.  (It seems
> that "base" packages are more likely to use a 'base-extension' naming
> pattern, vs. the 'namespace.project' pattern used by "pure" packages.)
>
> Of course, I am certainly not opposed to supporting base packages, and
> Martin's version of PEP 382 is a plus for setuptools because it would allow
> setuptools to better support the "base" scenario.
>
> But pure packages are definitely not a minority; in fact, a superficial
> observation of the full PyPI list suggests that there may be almost as many
> projects using pure-namespace packages, as there are non-namespaced
> projects!
>

In the survey I have done on packaging, 34% of the people that
answered are using setuptools namespace
feature, which currently makes it impossible to use the namespace for
the base package.

Now for the "base" or "core" package, what peoplethat uses setuptools
do most of the time:

1- they use zc.buildout so they don't need a base package : they list
in a configuration files all packages needed
   to build the application, and one of these package happen to have
the scripts to launch the application.

2 - they have a "main" package that doesn't use the same namespace,
but uses setuptools instal_requires metadata
     to include namespaced packages. It acts like zc.buildout in some ways.

For example, you mentioned atomisator.* in your example, this app has
a main package called "Atomisator" (notice the upper A)
that uses strategy #2

But frankly, the "base package" scenario is not spread these days
simply because it's not obvious to
do it without depending on a OS that has its own strategy to install packages.
For example, if you are not under debian, it's a pain to use logilab
packages because
they use this common namespace for several packages and a plain python
installation of the various packages
won't work out of the box under other systems like windows. (and for
pylint, I ended up creating my own distribution for windows...)

So :
- having namespaces natively in Python is a big win (Namespaces are
one honking great idea -- let's do more of those!)
- being able to still write some code under the primary namespace is
something I (and lots of people) wish we could do
  with setuptools, so it's a big win too.

Regards,
Tarek
-- 
Tarek Ziadé | http://ziade.org