David Cournapeau wrote:
Michael Hearne wrote:
I'm creating a Python application that is internal to my organization, but will be installed on both Linux and Mac OS X machines. The application depends heavily on a number of "non-pure" modules (those that have C/C++/FORTRAN components), like numpy, scipy, gdal, etc.
What is the most "pythonic" way to bundle these types of modules inside my application?
I've investigated dist_utils and setuptools, and I don't see an easy way with those tools to include build instructions for packages built with autconf tools.
Is my only recourse to write a custom install script that calls the "configure;make;make install" from a shell?
You have two problems: building and packaging. For packaging, autoconf will not help you much outside providing a coherence between packages, I think; distutils/setuptools have packaging tools, but I don't know how good they are.
On Linux, the packaging system depends on the distribution: since it is only internal, the problem is severely simplified, since you are likely to target only one format (rpm, deb, etc...). On Mac OS X, you have also tools to package .pkg and .mpkg (which are a set of .pkg), available freely on apple dev website; I have not used them much, but they look simple and easy to use for simple command line packages.
You could hack something in distutils to call configure and so on, but I don't think it will be pleasant. Building packages using distutils/setuptools is easy, and do not need much configuration. So I think it is more natural to use a top level tool calling distutils/setuptools than to use distutils as the top level tool. As a pythonic way, you may consider build tools, such as scons or waf, both written in python.
But frankly, I would not bother too much about pythonic way: since some tools will be shell based anyway (autotools, packaging tools), and you don't have to care about windows, using make and shell may actually be easier.
For the build part, you may take a look at my garnumpy package: it is essentially a set of rules for Gnu Makefiles, and it can build numpy + scipy, with a configurable set of dependencies (ATLAS, NETLIB BLAS/LAPACK, fftw, etc....). It can build both distutils and autotools-based packages.
http://www.ar.media.kyoto-u.ac.jp/members/david/archives/garnumpy/garnumpy-0...
I used it sucessfully on linux and cygwin, so I would say it should work on Mac OS X without too much trouble. The only thing which will be likely to be a pain is fat binaries (Universal). I use it to build a totally self contained numpy/scipy installation, which is a first step toward packaging. If you think it can be useful to you, don't hesitate to ask questions; there is also a bzr archive if you want to have access to the dev history of the tool
cheers,
David _______________________________________________ SciPy-user mailing list SciPy-user@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-user
Hi David, I am having a hell of a time with lapack/atlas. I already posted the issue I currently have with scipy after following the building doc in scipy web site (see http://projects.scipy.org/pipermail/scipy-user/2007-November/014506.html). Besides, octave build seems to also have serious issues with my lapack/atlas build. So I immediately downloaded youor package, but I seem to have a problem with the numpy download : make[3]: Leaving directory `/data1/sources/python/garnumpy-0.4/platform/numpy' # Change default path when looking for libs to fake dir, # so we can set everything by env variables cd work/main.d/numpy-1.0.3.1 && PYTHONPATH=/home/cohen/garnumpyinstall/lib/python2.5/site-packages:/home/cohen/garnumpyinstall/lib/python2.5/site-packages/gtk-2.0 /usr/bin/python \ setup.py config_fc --fcompiler=gnu config Traceback (most recent call last): File "setup.py", line 90, in <module> setup_package() File "setup.py", line 60, in setup_package from numpy.distutils.core import setup File "/data1/sources/python/garnumpy-0.4/platform/numpy/work/main.d/numpy-1.0.3.1/numpy/__init__.py", line 39, in <module> import core File "/data1/sources/python/garnumpy-0.4/platform/numpy/work/main.d/numpy-1.0.3.1/numpy/core/__init__.py", line 8, in <module> import numerictypes as nt File "/data1/sources/python/garnumpy-0.4/platform/numpy/work/main.d/numpy-1.0.3.1/numpy/core/numerictypes.py", line 83, in <module> from numpy.core.multiarray import typeinfo, ndarray, array, empty, dtype ImportError: No module named multiarray make[2]: *** [configure-custom] Error 1 make[2]: Leaving directory `/data1/sources/python/garnumpy-0.4/platform/numpy' make[1]: *** [../../platform/numpy/cookies/main.d/install] Error 2 make[1]: Leaving directory `/data1/sources/python/garnumpy-0.4/platform/scipy' make: *** [imgdep-main] Error 2 Any idea? Your package is anyway a great idea. I would actually love to see it work, of course, but also allow for the possibility to point to already exisiting source directories for numpy and scipy (for instance from a previous svn checkout). I did not read the doc of your package so it might actually be there.... anyway, thanks in advance for your help. Johann