[Python-Dev] PEP 382: Namespace Packages
glyph at divmod.com
glyph at divmod.com
Thu Apr 16 05:46:02 CEST 2009
On 15 Apr, 09:11 pm, pje at telecommunity.com wrote:
>I think that there is some confusion here. A "main" package or
>buildout that assembles a larger project from components is not the
>same thing as having a "base" package for a namespace package.
I'm certainly confused.
Twisted has its own system for "namespace" packages, and I'm not really
sure where we fall in this discussion. I haven't been able to follow
the whole thread, but my original understanding was that the PEP
supports "defining packages", which we now seem to be calling "base
packages", just fine. I don't understand the controversy over the
counterproposal, since it seems roughly functionally equivalent to me.
I'd appreciate it if the PEP could also be extended cover Twisted's very
similar mechanism for namespace packages,
"twisted.plugin.pluginPackagePaths". I know this is not quite as widely
used as setuptools' namespace package support, but its existence belies
a need for standardization.
The PEP also seems a bit vague with regard to the treatment of other
directories containing __init__.py and *.pkg files. The concept of a
"defining package" seems important to avoid conflicts like this one:
More specifically I don't quite understand the PEP's intentions towards
hierarchical packages. It says that all of sys.path will be searched,
but what about this case?
In Twisted, the suggested idiom to structure a project which wants to
provide Twisted plugins is to have a directory structure like this:
If you then put MyProject on PYTHONPATH, MyProject/twisted/plugins will
be picked up automatically by the plugin machinery. However, as
"twisted" is *not* a "namespace" package in the same way, .py files in
MyProject/twisted/ would not be picked up - this is very much
intentional, since the "twisted" namespace is intended to be reserved
for packages that we actually produce. If either MyProject/twisted or
MyProject/twisted/plugins/ had an __init__.py, then no modules in
MyProject/twisted/plugins/ would be picked up, because it would be
considered a conflicting package.
This is important so that users can choose not to load the system-
installed Twisted's plugins when they have both a system-installed
version of Twisted and a non-installed development version of Twisted
found first on their PYTHONPATH, and switch between them to indicate
which version they want to be the "base" or "defining" package for the
Developers might also want to have a system-installed Twisted, but a
non-installed development version of MyProject on PYTHONPATH.
I hope this all makes sense. As I understand it, both setuptools and
the proposed standard would either still have the bug described by
ticket 2339 above, or would ignore twisted/plugins/ as a namespace
package because its parent isn't a namespace package. I apologize for
not testing with current setuptools before asking, but I'm not sure my
experiments would be valid given that my environment is set up with
assumptions from Twisted's system.
P.S.: vendor packaging systems *ARE* a major use case for just about any
aspect of Python's package structure. I really liked MvL's coverage of
"vendor packages", in the PEP, since this could equally well apply to
MSIs, python libraries distributed in bundles on OS X, debs, or RPMs.
If this use-case were to be ignored, as one particular fellow seems to
be advocating, then the broken packages and user confusion that has been
going on for the last 5 years or so is just going to get worse.
More information about the Python-Dev