Re: [Python-Dev] PEP 382: Namespace Packages
At 10:00 PM 4/15/2009 +0200, Tarek Ziadé wrote:
Now for the "base" or "core" package, what peoplethat uses setuptools do most of the time:
1- they use zc.buildout so they don't need a base package : they list in a configuration files all packages needed to build the application, and one of these package happen to have the scripts to launch the application.
2 - they have a "main" package that doesn't use the same namespace, but uses setuptools instal_requires metadata to include namespaced packages. It acts like zc.buildout in some ways.
For example, you mentioned atomisator.* in your example, this app has a main package called "Atomisator" (notice the upper A) that uses strategy #2
I think that there is some confusion here. A "main" package or buildout that assembles a larger project from components is not the same thing as having a "base" package for a namespace package. A base or core package is one that is depended upon by most or all of the related projects. In other words, the dependencies are in the *opposite direction* from what you described above. To have a base package in setuptools, you would move the target code from the namespace package __init__.py to another module or subpackage within your namespace, then make all your other projects depend on the project containing that module or subpackage. And I explicitly excluded from my survey any packages that were following this strategy, on the assumption that they might consider switching to an __init__.py or __pkg__.py strategy if some version of PEP 382 were supported by setuptools, since they already have a "base" or "core" project -- in that case, they are only changing ONE of their packages' distribution metadata to adopt the new strategy, because the dependencies already exist.
So : - having namespaces natively in Python is a big win (Namespaces are one honking great idea -- let's do more of those!) - being able to still write some code under the primary namespace is something I (and lots of people) wish we could do with setuptools, so it's a big win too.
Yes, that's why I support Martin's proposal: it would allow setuptools to support this case in the future, and it would also allow improved startup times for installations with many setuptools-based namespace packages installed in flat form. (Contra MAL's claims of decreased performance: adopting Martin's proposal allows there to be *fewer* .pth files read at startup, because only .pkg files for an actually-imported package need to be read.)
On 15 Apr, 09:11 pm, pje@telecommunity.com wrote:
I think that there is some confusion here. A "main" package or buildout that assembles a larger project from components is not the same thing as having a "base" package for a namespace package.
I'm certainly confused. Twisted has its own system for "namespace" packages, and I'm not really sure where we fall in this discussion. I haven't been able to follow the whole thread, but my original understanding was that the PEP supports "defining packages", which we now seem to be calling "base packages", just fine. I don't understand the controversy over the counterproposal, since it seems roughly functionally equivalent to me. I'd appreciate it if the PEP could also be extended cover Twisted's very similar mechanism for namespace packages, "twisted.plugin.pluginPackagePaths". I know this is not quite as widely used as setuptools' namespace package support, but its existence belies a need for standardization. The PEP also seems a bit vague with regard to the treatment of other directories containing __init__.py and *.pkg files. The concept of a "defining package" seems important to avoid conflicts like this one: http://twistedmatrix.com/trac/ticket/2339 More specifically I don't quite understand the PEP's intentions towards hierarchical packages. It says that all of sys.path will be searched, but what about this case? In Twisted, the suggested idiom to structure a project which wants to provide Twisted plugins is to have a directory structure like this: MyProject/ myproject/ __init__.py twisted/ plugins/ myproject_plugin.py If you then put MyProject on PYTHONPATH, MyProject/twisted/plugins will be picked up automatically by the plugin machinery. However, as "twisted" is *not* a "namespace" package in the same way, .py files in MyProject/twisted/ would not be picked up - this is very much intentional, since the "twisted" namespace is intended to be reserved for packages that we actually produce. If either MyProject/twisted or MyProject/twisted/plugins/ had an __init__.py, then no modules in MyProject/twisted/plugins/ would be picked up, because it would be considered a conflicting package. This is important so that users can choose not to load the system- installed Twisted's plugins when they have both a system-installed version of Twisted and a non-installed development version of Twisted found first on their PYTHONPATH, and switch between them to indicate which version they want to be the "base" or "defining" package for the twisted/plugins/ namespace. Developers might also want to have a system-installed Twisted, but a non-installed development version of MyProject on PYTHONPATH. I hope this all makes sense. As I understand it, both setuptools and the proposed standard would either still have the bug described by ticket 2339 above, or would ignore twisted/plugins/ as a namespace package because its parent isn't a namespace package. I apologize for not testing with current setuptools before asking, but I'm not sure my experiments would be valid given that my environment is set up with assumptions from Twisted's system. P.S.: vendor packaging systems *ARE* a major use case for just about any aspect of Python's package structure. I really liked MvL's coverage of "vendor packages", in the PEP, since this could equally well apply to MSIs, python libraries distributed in bundles on OS X, debs, or RPMs. If this use-case were to be ignored, as one particular fellow seems to be advocating, then the broken packages and user confusion that has been going on for the last 5 years or so is just going to get worse.
participants (2)
-
glyph@divmod.com
-
P.J. Eby