[Distutils] zc.buildout fails to use system-installed dep?

[Barry Warsaw]

To start with, the Debian/Ubuntu arrangement is made even more confusing for
there being three competing "build systems" for Python.  There are the
traditional python-central and python-support systems, which most packages
these days use.  python-support is probably more popular than python-central.
Both however are being obsoleted for the new goodness, dh_python2.  The
biggest benefit to dh_python2 is that the symlinks are created at build-time
and appear in the package, instead of being created at install-time and
managed by what can be flaky post-scripts.  The conversion process will begin
happening in earnest once Debian squeeze is released.

Why do we have symlinks in the first place?  It's because Debian and Ubuntu
support multiple active versions of Python at the same time.  Once Python 2 is
killed off and we're all using >= Python 3.2, we can get rid of even this
cruft.  That's the major reason why I worked on PEPs 3147 and 3149.

On Dec 08, 2010, at 06:32 PM, Marius Gedminas wrote:

>On Wed, Dec 08, 2010 at 12:21:55PM +0100, Reinout van Rees wrote:
>> On 12/05/2010 11:14 PM, Alan Franzoni wrote:
>> >I've verified this happens all the times on Ubuntu 10.10, both 32 and
>> >64 bit. I don't know if it's Ubuntu's or zc.buildout's fault; I've
>> >opened a ticket.
>> 
>> I have the same problem on ubuntu.  I never managed to find out what
>> the real reason is, but my guess is that it's ubuntu's way of
>> installing python packages for multiple python versions.  You have a
>> /usr/lib/python2.6/dist-packages/ directory,
>
>which is on sys.path
>
>> but also a
>> /usr/lib/pyshared/python2.6/,
>
>which isn't.
>
>But then there's /usr/lib/pymodules/python2.6, which is on sys.path,
>and I've no idea what it's about.  dist-packages/python-support.pth adds
>this one.

All roads point to pyshared, which is where the files are *actually*
installed.  pymodules and dist-packages point into pyshared and are there to
handle the multiple-installed-python arrangement.  I think the difference
comes about depending on whether python-central or python-support is being
used.

>> for instance.  Something somewhere is
>> doing some funny sys.path mangling, probably ubuntu.
>
>Debian and Ubuntu use dist-packages instead of site-packages to avoid
>conflicts with manual installations (I'm not sure if they mean manual as
>in sudo easy_install foo, or if they mean manual as in self-compiled
>Pythons.  Probably the latter.).

Yes.  A from-source build and install of Python would use
/usr/local/lib/pythonX.Y/site-packages, but Debian's interpretation of the FHS
puts that directory under the domain of the system administrator.  Which
means, it was possible to install third party packages into a from-source
build of Python (with default configure options and 'make altinstall') and
break your system Python, or vice versa.  dist-packages is a compromise to the
decade's tradition of Python source default locations.  (Arguably, a
from-source build of Python with the default options should install into /opt.)

>If zc.buildout assumes it can find system packages in site-packages
>instead of dist-packages, then it would fail.  I've no idea if that's
>actually the case.

I don't remember specifically for zc.buildout, but distribute for example has
been modified to use dist-packages.

>/usr/share/doc/python/python-policy.txt.gz has more information about
>Debian and Ubuntu specifics.
>
>> The end result is that I still haven't gotten buildout 1.5.x to work
>> on ubuntu with the system packages that I need.  I'm still at 1.4.4
>> with the custom 1.4-x-only bootstrap, in combination with
>> osc.recipe.sysegg
>
>IIRC you can use the new bootstrap and ask it to get a specific version
>of zc.buildout.  I never needed recipes to use system Python packages in
>1.4 buildouts, but that was before Python 2.6 and dist-packages --
>Debian's Python 2.5 still uses site-packages.

I'm using zc.buildout 1.5.2 on Ubuntu 10.10 for Mailman 3.  There is a known
bug related to namespace packages, which force you to use
include-site-packages=false in your buildout.cfg.  Here's the bug report:

https://bugs.launchpad.net/zc.buildout/+bug/659231

Gary is assigned to the bug but there is as yet no resolution.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20101208/060382d2/attachment-0001.pgp>