[Numpy-discussion] Announcing toydist, improving distribution and packaging situation

[David Cournapeau]

On Tue, Dec 29, 2009 at 3:49 AM, Dag Sverre Seljebotn
<dagss at student.matnat.uio.no> wrote:

>
> Do you here mean automatic generation of Ubuntu debs, Debian debs, Windows
> MSI installer, Windows EXE installer, and so on? (If so then great!)

Yes (although this is not yet implemented). In particular on windows,
I want to implement a scheme so that you can convert from eggs to .exe
and vice et versa, so people can still install as exe (or msi), even
though the method would default to eggs.

> If this is the goal, I wonder if one looks outside of Python-land one
> might find something that already does this -- there's a lot of different
> package format, "Linux meta-distributions", "install everywhere packages"
> and so on.

Yes, there are things like 0install or autopackage. I think those are
deemed to fail, as long as it is not supported thoroughly by the
distribution. Instead, my goal here is much simpler: producing
rpm/deb. It does not solve every issue (install by non root, multiple
// versions), but one has to be realistic :)

I think automatically built rpm/deb, easy integration with native
method can solve a lot of issues already.

>
>  - Currently I'm making a Sage SPKG for CHOLMOD. This essentially gets the
> job done by not bothering about the problem, not even using the
> OS-installed Python.
>
> Something that would spit out both Sage SPKGs, Ubuntu debs, Windows
> installers, both with Python code and C/Fortran code or a mix (and put
> both in the place preferred by the system in question), seems ideal. Of
> course one would still need to make sure that the code builds properly
> everywhere, but just solving the distribution part of this would be a huge
> step ahead.

On windows, this issue may be solved using eggs: enstaller has a
feature where dll put in a special location of an egg are installed in
python such as they are found by the OS loader. One could have
mechanisms based on $ORIGIN + rpath on linux to solve this issue for
local installs on Linux, etc...

But again, one has to be realistic on the goals. With toydist, I want
to remove all the pile of magic, hacks built on top of distutils so
that people can again hack their own solutions, as it should have been
from the start (that's a big plus of python in general). It won't
magically solve every issue out there, but it would hopefully help
people to make their own.

Bundling solutions like SAGE, EPD, etc... are still the most robust
ways to deal with those issues in general, and I do not intended to
replace those.

> What I'm saying is that this is a software distribution problem in
> general, and I'm afraid that Python-specific solutions are too narrow.

Distribution is a hard problem. Instead of pushing a very narrow (and
mostly ill-funded) view of how people should do things like
distutils/setuptools/pip/buildout do, I want people to be able to be
able to build their own solutions. No more "use this magic stick v
4.0.3.3.14svn1234, trust me it work you don't have to understand"
which is too prevalant with those tools, which has always felt deeply
unpythonic to me.

David