[Python-Dev] PEP 370, open questions
glyph at divmod.com
glyph at divmod.com
Thu Jan 17 09:40:04 CET 2008
On 07:55 am, lists at cheimes.de wrote:
>A CC of the mail goes to the authors of setuptools, virtual python,
>working env and virtual env. What's your opinion on the PEP? Do you
>some input and advice for me?
I wrote a similar tool called Combinator
(http://divmod.org/trac/wiki/DivmodCombinator) which Divmod uses quite
intensively. I think a significant portion of the Twisted community
also uses it for development. Luckily you don't need to CC me specially
as I'm here already ;-). One of the features it provides is a user
site-packages directory much in the same style that the PEP proposes, so
I'll comment on my experiences maintaining it.
>The PEP 370 (http://www.python.org/dev/peps/pep-0370) "per user site
>packages directory" has several open questions:
>* Are the directories for Windows, Mac and Unix fine?
>* Mac: Should framework and non-framework builds of Python use
> the same directories?
One of the problems we've encountered over and over again with
Combinator is that MacOS has a dual personality. Its personality is not
just an issue of framework vs. non-framework build, but a fundamental
question of "is this platform a UNIX or is it not a UNIX".
An example: Let's say you have a GNU autotools project in C, which we'll
call "Xxx", and a set of Python bindings, PyXxx. Combinator deals with
this by using ~/.local, and providing scripts to set up PATH and
LD_LIBRARY_PATH to point to ~/.local/bin and ~/.local/lib, respectively.
I'm not suggesting that Python come with such a tool (although it might
be handy, at least until this convention catches on with distributors),
but it should at least make it clear how one would achieve the following
./configure --prefix ~/.local
python setup.py install --prefix ~/.local
Using Combinator, the user is now fully set up to "import xxx". But
only if they're using Linux, because I made the same mistake (which we
will probably be correcting at some point soon) of using Python's
*existing* per-user site-packages directory of ~/Library/Python on the
mac, and not adding ~/.local.
On the Mac, our user now has a problem: given that ~/Library/Python/
doesn't follow the /usr or /usr/local style filesystem layout,
autotools-based projects will not build their libraries in the right
places using a single command-line option. (I guess maybe you could do
something with --bindir and --sbindir and --exec-prefix or whatever, I
don't know. I tend to get bored and wander off when I have to pass more
than 3 options to a configure script.)
A user familiar with the Mac alone might know what to do at this point
(to be honest, I don't!), but I do know that people familiar with both
platforms are confused by this apparently gratuitous inconsistency.
They follow familiar procedure on a Linux box, it works great, then they
do the same thing on a Mac (with the same shell, an apparently similar
version of Python) and it doesn't. Keep in mind that "./configure
--prefix" is familiar procedure from a lot more places than Combinator.
For example, on shared hosting where I didn't have root, I've used this
same trick without Combinator, building Python itself with --prefix
~/.local and editing .bashrc to modify the appropriate env vars.
My suggestion would be to have *both* ~/.local *and* ~/Library/Python be
honored on the Mac, because there really isn't much harm in doing so.
Perhaps it would make sense for non-framework builds to not honor
~/Library/Python, but I am pretty certain, based on experience fielding
requests from confused users, that framework builds should still honor
Windows has this problem less because everything has to be done so
In any event, the really important thing from my perspective is that the
PEP should explain how this very common use-case for per-user
installation of libraries can be accomplished on each of the "big three"
platforms. This explanation should be put somewhere that users can
easily find it when they are building libraries.
I don't know what the "right way" to do this on Windows is though, so I
can't offer much help there. Something to do with MinGW and intense
suffering, I would guess.
>* The patch also adds a usecustomize hook to site. Is it useful and
> should it stay?
Should this be "usercustomize"? I thought it was a typo but I see the
same typo in the PEP. I have often wished for something like this for
debugging, and it might have other uses, but there should be a caution
against abuse :).
More information about the Python-Dev