[Numpy-discussion] [matplotlib-devel] Announcing toydist, improving distribution and packaging situation
cournape at gmail.com
Sun Jan 3 07:23:14 EST 2010
On Sun, Jan 3, 2010 at 8:05 PM, Nathaniel Smith <njs at pobox.com> wrote:
> On Tue, Dec 29, 2009 at 6:34 AM, David Cournapeau <cournape at gmail.com> wrote:
>> Buildout, virtualenv all work by sandboxing from the system python:
>> each of them do not see each other, which may be useful for
>> development, but as a deployment solution to the casual user who may
>> not be familiar with python, it is useless. A scientist who installs
>> numpy, scipy, etc... to try things out want to have everything
>> available in one python interpreter, and does not want to jump to
>> different virtualenvs and whatnot to try different packages.
> What I do -- and documented for people in my lab to do -- is set up
> one virtualenv in my user account, and use it as my default python. (I
> 'activate' it from my login scripts.) The advantage of this is that
> easy_install (or pip) just works, without any hassle about permissions
It just works if you happen to be able to build everything from
sources. That alone means you ignore the majority of users I intend to
No other community (except maybe Ruby) push those isolated install
solutions as a general deployment solutions. If it were such a great
idea, other people would have picked up those solutions.
> This should be easier, but I think the basic approach is sound.
> "Integration with the package system" is useless; the advantage of
> distribution packages is that distributions can provide a single
> coherent system with consistent version numbers across all packages,
> etc., and the only way to "integrate" with that is to, well, get the
> packages into the distribution.
Another way is to provide our own repository for a few major
distributions, with automatically built packages. This is how most
open source providers work. Miguel de Icaza explains this well:
I hope we will be able to reuse much of the opensuse build service
> On another note, I hope toydist will provide a "source prepare" step,
> that allows arbitrary code to be run on the source tree. (For, e.g.,
> cython->C conversion, ad-hoc template languages, etc.) IME this is a
> very common pain point with distutils; there is just no good way to do
> it, and it has to be supported in the distribution utility in order to
> get everything right. In particular:
> -- Generated files should never be written to the source tree
> itself, but only the build directory
> -- Building from a source checkout should run the "source prepare"
> step automatically
> -- Building a source distribution should also run the "source
> prepare" step, and stash the results in such a way that when later
> building the source distribution, this step can be skipped. This is a
> common requirement for user convenience, and necessary if you want to
> avoid arbitrary code execution during builds.
Build directories are hard to implement right. I don't think toydist
will support this directly. IMO, those advanced builds warrant a real
build tool - one main goal of toydist is to make integration with waf
or scons much easier. Both waf and scons have the concept of a build
directory, which should do everything you described.
More information about the NumPy-Discussion