[Distutils] [Python Language Summit] Distutils / Packaging survey

[David Cournapeau]

Ian Bicking wrote:
> On Sat, Jan 31, 2009 at 2:40 AM, David Cournapeau
> <david at ar.media.kyoto-u.ac.jp <mailto:david at ar.media.kyoto-u.ac.jp>>
> wrote:
>
>     Ian Bicking wrote:
>     > On Fri, Jan 30, 2009 at 12:39 PM, Floris Bruynooghe
>     > <floris.bruynooghe at gmail.com
>     <mailto:floris.bruynooghe at gmail.com>
>     <mailto:floris.bruynooghe at gmail.com
>     <mailto:floris.bruynooghe at gmail.com>>> wrote:
>     >
>     >     I imagine things like libdir, prefix, datadir, docdir and
>     other things
>     >     copied from autoconf.  Where the defaults would be something
>     like:
>     >
>     >     prefix = sys.prefix
>     >     libdir = sys.prefix/lib/pythonX.Y/site-packages/pkgname
>     >     datadir = sys.prefix/share/mypackage
>     >     docdir = sys.prefix/share/doc/mypackage
>     >
>     >
>     > I wouldn't want to use those.  What goes in libdir, what goes in
>     > datadir?  I don't know, and frankly the distinctions start getting
>     > really arbitrary.
>
>     They are not arbitrary - they come from standard usage and have a
>     rationale, at least on Unix (datadir for arch independent, and libdir
>     for arch dependent, to simplify).
>
>
> I'm just about ready to run screaming from this discussion... so no, I
> want no part of determining what the "right" place for these files is,
> nor do I find it self-evident.

I never said it was self-evident. And about the part of determining what
the right place is: the whole point is that you, as a developer, won't
have to care :) Only by changing the distutils implementation and adding
some options to the install command, but without changing *anything* in
the setup.py, would many if not most python softwares be easier to
install along the FHS. And where something is missing, the *packager*
could send a patch to fix some metadata upstream, instead of keeping
their changes separately.

If you don't have any experience with autotools, it is a bit hard to go
into details, but you almost never care about what goes where for the
most part. You just tag some files as programs, some as docs, etc...
Most of it being implicit.

>
> Sorry, I didn't describe what I meant.
>
> I imagine some file like package-data.conf, containing:
>
> data mypackage/templates/
> docs docs/_build/
>
> At least in this example the first word is some tag, and the second is
> the directory, or files, or maybe a wildcard or something determining
> what files that tag applies to.  Everything not declared but present
> in a package (or as a module) would default to "lib", and everything
> outside of that (like the setup.py file) would be "ignore".

I think this does not have to be as complicated. We already need to tag
some files in the setup.py file, so we could do:

from distutils.core import setup, Extension

setup(name='example',
        version='1.0',
        description='Example',
        author='You',
        author_email='me at python.net',
        packages=['example', 'example.foo'],
        ext_modules=[Extension('example.foo._bar', ['example/foo/bar.c'])],
        pdfdoc=['doc/doc.pdf'],
        htmldoc=['doc/html'],
        pacakge_data=['example/data/data1.dat', 'example/data/data2.dat']
        )

Note that except pdfdoc and htmldoc, all this is already there in
distutils. Then, you could do

python setup.py install --prefix=tmp

ending up in (on Linux):

tmp/lib/python-$PYVER/site-packages/example/__init__.py
...
tmp/lib/python-$PYVER/site-packages/example/foo/bar/_bar.so
tmp/lib/python-$PYVER/site-packages/example/data/data1.dat
...
tmp/lib/python-$PYVER/site-packages/example/doc/doc.pdf
...

As is currently done. But you could also do:

python setup.py install --libdir=tmp/lib/ --datadir=tmp/share/
--pdfdoc=tmp/share/doc

which would result in:

tmp/share/example/__init__.py
...
tmp/lib/python-$PYVER/site-packages/example/foo/bar/_bar.so
tmp/share/example/data/data1.dat
...
tmp/share/doc/example/doc.pdf
...

I did not try to follow any kind of Debian policy - the point is that
you can decide how to do the mapping category -> location at install
time. And if a mapping is missing/buggy, the packager can fix it, and
send you a patch. You, as the upstream developer, would have nothing to
do - actually, I certainly expect most people not to care at all about
all this, and they will just have to wait for patchs to come in (that's
the optimistic view :) ).

cheers,

David