[Distutils] more thoughts on python package management

kiorky kiorky at cryptelium.net
Tue Sep 30 22:11:34 CEST 2008

Chris Withers a écrit :
> > Hi All,
> >
> > I've been trying to catch up on all the packaging discussions but
> > couldn't find the right place to reply so thought I'd just do so
> > seperately...
Maybe, we have all our own definition ...
> >
> > Probably the biggest thing that strikes me now is that
> > distutils/setuptools/distribute/pacman/whatever should aim to do much
> > less...
I don't totally agree, we must provide a tool to build from start to end our
package even if we let means (eg via complete metadatas) to do the job in
another way, by other tools.
> >
> > In fact, I get the feeling what we really need is a way for package
> > maintainers to provide the following metadata:
> >
> > - where the docs are
> >
> > - where the tests are and how they're run
> >
> > - how anything not-python should be built
> >
> > - what the dependencies are
> >   (maybe even what the non-python dependencies are!)
> >
> > - what version of the package this is
> >
> > This should be in a build-tool independent fashion such that any build
> > tools, but especially those of operating system maintainers, can run
> > over the same metadata and build their packages.
+1 for more explicit metadata on python packages,
But i don't think our target is only the target "OS" and its relative packages
manager. I really can't see how to get running a lot of projects with
inter-related incompatibilities on the same system without some sort of isolation
So we cannot say we will be able to install whatever we want with everything
else at the same time, system wide. And selecting versions by hand, in each
project, can be painful. I'd prefer to have a "project" or "environment
specific" approach like buildout, or virtualenv, or the combination of these
tools, even maybe a wrap-up tool like minitage [1].

> >
> > The only other critical thing for me is that *all* of the above metadata
> >  should be available post-install.
> >
I 'd prefer to know what i am installing before installing it. For me, the
metadata are predictable things, which must be there before installing any part
of a project.

> > With the above in place, we free up the evolution of build tools and let
> > the OS-specific packaging tools play nicely.
> >
> > I think a good aim would also be to have some "one-way-to-do-it" python
> > tools too for:
> >
> > - installing a package
> >
> > - uploading a package to PyPI
> >
> > - getting a package from PyPI
> >
> >
+1, but see under
> > ...without any silly big plugin system in the way distutils currently
> > works.
> >
> > What do other people feel?
> >
> > cheers,
> >
> > Chris
> >

IMHO, a good management suite for python packages is a collection of tools,
plugins and API bindings which:
 - Can do all its job in total isolation (installation in /prefix even where our
only privilege is a filesystem write access)
   There two levels i can see there:
     - Isolation from the OS
     - Isolation in the "target environment" (/package1/the-stuff and
 - Is OS independant (maybe the most hard thing to do)
 - Is failure tolerant:
    - Handle offline mode
    - Know how to go backward
    - Use a sandbox before (un)installing
 - Let us means to add, remove and override the packages 's repositories and
even the packages themselves.
 - Can use a download cache
 - Deal with dependencies
 - Can distribute the package
 - Can fetch the package
 - Can build the package
 - Can Test the package
 - Can install the package from source or binary
 - Trace and store what is installed
 - Can uninstall the package
 - Know how to deal with reverse dependencies problems, even if this is a tool
with just rebuild them like the gentoo's ones [2].
 - Provide goods metadata
 - Have all the classical features cli tools like apt, emerge ... have:
 - Search on installed and available softwares
 - List installed files
 - Knows from which package a file is belonging to
 - Have an API to let us build, query, distribute, get, ... our packages.
 - Have some hook/plugin registration system like the egg's entry points.

And the according packages's metadata must contain:
 - package name
 - package version
 - Detailed dependencies requirements which include also incompatibilities (like
OS ones)
 - Maybe some knobs system like the USES in gentoo.
 - where we get the package
 - The way we build/install/uninstall it
 - Author/website/license and so on
 - Where are the tests
 - How the tests are run
 - hooks registration information

There is nothing new there, Those are classical package manager needs. Maybe
more "source packages manager" needs because we are dealing with something which
can be built on the target. What is different there is that i try  introduce an
"environment" approach more than a "system centric" one.

[1] http://www.minitage.org/doc/rst
 - emerge @preserved-rebuild:
 - revdep-rebuild : http://gentoo-wiki.com/TIP_Control_revdep-rebuild

GPG Key FingerPrint: 0x1A1194B7681112AF

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 260 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20080930/e8df7134/attachment-0001.pgp>

More information about the Distutils-SIG mailing list