Features for a Python package manager?
rkern at ucsd.edu
Sun Dec 26 09:42:00 CET 2004
Georg Brandl wrote:
> Robert Kern wrote:
>>Mike Meyer wrote:
>>>Nick Coghlan <ncoghlan at iinet.net.au> writes:
>>>>I don't know enough about Portage to answer that question. I do know
>>>>any package manager which made it into the standard distribution would
>>>>need to work for at least the big three platforms (Windows/Mac/*nix) :)
>>>Being written in python - and hopefully integrated into Distutils -
>>>why shouldn't it work on any platform that Python worked on?
>>Assumptions about directory structures, and the like.
> That is already taken care by the distutils.
More or less. I'm in the middle of packaging up ~40 Python packages for
the Mac. For a "standard" packaging mechanism, distutils allows for
some bloody idiosyncratic ways to say "put these files there". This is a
hard problem, and it's not solved entirely by distutils.
I don't think anyone has satisfactorily solved the problem of
distributing data with libraries. Well, okay, the *distribution* isn't
the problem. Having the library be able to locate that data on all
platforms is difficult. Some packages will more-or-less hardcode
*nix-type paths which may be inappropriate even on some *nix-type
platforms (yes, PyX, I'm looking at you :-)). A general package system
like Portage has the freedom of being able to dictate these things. A
Python package manager does not. You can establish a standard for each
of the Big Three platforms, but it may not do you much good if the
libraries don't know about it.
CPAN is a closer analogue in this regard and would probably be a better
tool to study and copy from than Portage. I don't know much about it,
but how it responds to these issues will probably more instructive than
how Portage does.
You also have problems with distributing non-Python dependencies like
libraries. You can sometimes punt and require the user to have already
installed said libraries and headers to compile against. This will
probably only work for source distributions on *nixes. You can probably
get away with appropriately placed DLLs on Windows. Macs may be trickier
(we Mac users are fussy :-)).
Exercise: VTK with its Python and TCL libraries (why TCL? It's necessary
to use VTK with Tkinter, which the premier VTK-Python application,
MayaVi, uses) should package up cleanly for all 3 main platforms.
External libraries *and* data! It's a joy!
Some other "problem packages" to practice on: PyX, ReportLab, PIL,
 In the sense that they are, to some extent, not painless to package
up in a very neat way on multiple platforms for one reason or another.
These are all very fine packages; packaging is a hard problem; and I'm
grateful for how well they *do* package up so far.
> The various PEPs already
> describe a simple method how to store package metadata, and I will try
> to follow these standards as close as possible. Of course, the PyPI
> would have to be adapted for that.
Fair enough. What parts of Portage do you intend to steal, then?
Please don't read any of this as discouragement. Python package
management desperately needs a champion, and I'm very glad to see
someone tackling these issues. I for one fully intend to use as much of
what you produce as I can to make MacEnthon fully upgradable.
rkern at ucsd.edu
"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
More information about the Python-list