[Distutils] zc.buildout fails to use system-installed dep?

Brian Sutherland brian at vanguardistas.net
Fri Dec 10 15:06:53 CET 2010

On Fri, Dec 10, 2010 at 08:19:32AM -0500, Barry Warsaw wrote:
> On Dec 10, 2010, at 01:42 PM, Brian Sutherland wrote:
> >On Fri, Dec 10, 2010 at 07:17:16AM -0500, Barry Warsaw wrote:
> >> The way zope.interface "owns" the zope namespace is not correct.  Ideally,
> >> there would be a separate binary package that owns zope/__init__.py and on
> >> which all other zope subpackages depends.  This could of course be built
> >> from the zope.interface source package.
> >
> >I originally had it that way, but was strongly advised to change it to
> >the current method to be able to pass the Debian FTP Master gauntlet.
> Just goes to illustrate the myth of TOOWTDI. :) AFAICT, from my discussions
> with various folks on debian-python, it should now[*] be done with the
> separate binary package.
> [*] At least until dh_python2, if we can ensure it always DTRT.

I'll just till dh_python2 then before changing anything ;)

hmm, perhaps I should join debina-python again...


> >> Second, we're going to make a big push after Squeeze is released to convert
> >> packaging to use dh_python2, the new goodness in Debian Python packaging.
> >
> >I've had a brief look at it already and will have a much deeper look
> >once squeeze is released. AFAIKR the only feature it was missing to
> >completely cover my usecase was good handling of setuptools extras.
> Can you be more specific?  Before I got swamped with the Python 2.7 transition
> (it will be the default in Ubuntu 11.04), I began looking at dh_python2,
> adding unit tests, etc.  I do plan to get back to it once we have the archive
> more happily on Python 2.7[*].
> [*] https://bugs.launchpad.net/ubuntu/+bugs?field.tag=python27

I always use zope.component as an example because it has the worst case
use of setuptools extras which a lot of other packages depend on.

In 3.9.1 this is:

    extras_require = dict(
        hook = ['zope.hookable'],
        persistentregistry = ['ZODB3'],
        zcml = ['zope.configuration',
        test = ['ZODB3',
        docs = ['z3c.recipe.sphinxdoc'],

The way we treat these extras in the python-zope.* packages is as if
they were extra binary packages with names like
python-${distribution_name}-${extra_name}. So zope.component is actually
6 binary packages:


But, we can't make 6 binary packages or the FTP Masters will kill us. So
we do a few things with extras:

    * Have Ignore them and their dependencies.
    * Provide the extra package from the main package and depend on it's
    * Have their dependencies as "suggests" on the main package
    * Break them out into a binary metapackage.

python-zope.component does all of these for it's extras as appropriate
in each case.

I wrote a plugin to debhelper 7 so this can be specified in an easy way
using environment variables in the rules file:


The code to do the work is in the python-van.pydeb package. One of the
things I had hoped to do after squeeze was port the van.pydep tests to
dh_python2 and then bother someone else to fix them.

python-zope.security is also another tough case. In that case, Ubuntu
needs to carry a diff against Debian to cope with the "untrustedpython"


> >> This should make things more consistent, and, while I have not yet tracked
> >> down specific details yet, it should make handling namespace packages much
> >> more robust.  For example, with my flufl packages, there is no binary
> >> package owning flufl/__init__.py.  That file does not show up in the binary
> >> package and yet it still gets created properly when any of the subpackages
> >> get installed.
> >
> >Any idea as to the mechanism?
> Hints, but nothing definite.  I'm not sure which part of the tool chain is
> suppressing the neamespace package's __init__.py, and which part is laying it
> down when the package gets installed.  It *is* nice that the packager doesn't
> have to worry about it though.  I don't think you should have to do the tricks
> zope.interface is doing, or even do the extra binary package.  IOW, it should
> Just Work.

+100 to just working :)
> >What about 2nd level namespace packages (horror: zope.app.foo)?
> Haven't tried that yet.

Well, cross that bridge when we come to it...

> >> Sadly PEP 382 was not complete in time for Python 3.2.
> >
> >Yeah, there are a lot of packaging related PEPs coming out lately. It's
> >really great to see attention being paid to these dusty corners:)
> Indeed!  I think we all owe Tarek and the other folks an unlimited supply of
> beers at the next Pycon. :)

Yep :)

Brian Sutherland

More information about the Distutils-SIG mailing list