I mailed this to Greg, and he said to try here:
I've got a project with a python wrapper/extension module (amoung others), called ClearSilver http://www.clearsilver.net/
The package uses autoconf/configure to determine which programs are available, and how to build things, etc. This includes external libaries and locations for them, like -lz, -ldb, etc.
Until recently, I've been just providing a Makefile for compiling the python extension, instead of using distutils. I don't really want to deal with compiling the shared library on all of the platforms, though, so I figured I'd go and create the setup.py script.
The trouble is, how to pass the information from configure to the setup.py. I was just going to pass them on the command line, ie from Makefile:
$(PYTHON) setup.py ext_build $(INCLUDES) $(LIBDIRS) $(LIBS)
where $(INCLUDES) is a list of -Ipaths and $(LIBDIRS) is a list of -Lpaths and $(LIBS) is a list of -llibraries.
Unfortunately, the command-line processing code, and the code for the setup.cfg file, seems to take the command line arguments and put them into a dictionary. Later, that dictionary is used to set the attributes of the ext_build class, which then translates the string into a single item list.
So, this means I can't specify more than one option on the command line and get a list of libraries or paths. In fact, there seems to be no way to set a list on an option via the command line. That seems bad. Most people seem to deal with this problem by having as much smarts in the setup.py file as there are in their configure script... which seems annoying, especially since if there are similar macros for finding libraries and the like in distutils, they are at least not documented.
Its also not clear to me how I can parse the command line arguments in setup.py... though I guess I could probably just import sys, run through sys.argv removing my arguments, or some such. I didn't try that, though. Instead, my "solution" is to parse the Makefile from the setup.py file, and setup the options from what I get. Obviously I can't write all of make, but it works well enough for my task (its not up there yet, it'll be in the next version).
Any thoughts on fixing the command line arguments to support list options?
Brandon
Il lun, 2003-08-04 alle 08:47, Brandon Long ha scritto:
I mailed this to Greg, and he said to try here:
I've got a project with a python wrapper/extension module (amoung others), called ClearSilver http://www.clearsilver.net/
The package uses autoconf/configure to determine which programs are available, and how to build things, etc. This includes external libaries and locations for them, like -lz, -ldb, etc.
Until recently, I've been just providing a Makefile for compiling the python extension, instead of using distutils. I don't really want to deal with compiling the shared library on all of the platforms, though, so I figured I'd go and create the setup.py script.
same problem here. i solved it by using configure to generate setup.py from setup.py.in (in setup.py you can specify multiple include paths, etc.) but i consider this to be an hack and a sub-optimal solution. the best would be to generate setup.cfg and let the user do last-minute adjustements directly to it.
Most of the configuration machinery is already available in distutils (via "python setup.py config"), so you may not need to run autoconf/configure at all. In one of my software projects, I was able to get rid of the autoconf/configure stuff altogether by using the corresponding distutils routines. These are much easier to use than autoconf/configure, and also guarantees that you are using the same compiler and compiler options for the config stage and the build stage. For some examples of "python setup.py config" (including finding external libraries and such), see
http://bonsai.ims.u-tokyo.ac.jp/~mdehoon/software/python/pygist.html
or
http://bonsai.ims.u-tokyo.ac.jp/~mdehoon/software/python/statistics.html
One caveat: there were some bugs in distutils' config machinery in Python 2.2. These have been fixed in Python 2.3.
--Michiel.
Brandon Long wrote:
I mailed this to Greg, and he said to try here:
I've got a project with a python wrapper/extension module (amoung others), called ClearSilver http://www.clearsilver.net/
The package uses autoconf/configure to determine which programs are available, and how to build things, etc. This includes external libaries and locations for them, like -lz, -ldb, etc.
Until recently, I've been just providing a Makefile for compiling the python extension, instead of using distutils. I don't really want to deal with compiling the shared library on all of the platforms, though, so I figured I'd go and create the setup.py script.
The trouble is, how to pass the information from configure to the setup.py. I was just going to pass them on the command line, ie from Makefile:
$(PYTHON) setup.py ext_build $(INCLUDES) $(LIBDIRS) $(LIBS)
where $(INCLUDES) is a list of -Ipaths and $(LIBDIRS) is a list of -Lpaths and $(LIBS) is a list of -llibraries.
Unfortunately, the command-line processing code, and the code for the setup.cfg file, seems to take the command line arguments and put them into a dictionary. Later, that dictionary is used to set the attributes of the ext_build class, which then translates the string into a single item list.
So, this means I can't specify more than one option on the command line and get a list of libraries or paths. In fact, there seems to be no way to set a list on an option via the command line. That seems bad. Most people seem to deal with this problem by having as much smarts in the setup.py file as there are in their configure script... which seems annoying, especially since if there are similar macros for finding libraries and the like in distutils, they are at least not documented.
Its also not clear to me how I can parse the command line arguments in setup.py... though I guess I could probably just import sys, run through sys.argv removing my arguments, or some such. I didn't try that, though. Instead, my "solution" is to parse the Makefile from the setup.py file, and setup the options from what I get. Obviously I can't write all of make, but it works well enough for my task (its not up there yet, it'll be in the next version).
Any thoughts on fixing the command line arguments to support list options?
Brandon
Il lun, 2003-08-04 alle 10:11, Michiel Jan Laurens de Hoon ha scritto:
Most of the configuration machinery is already available in distutils (via "python setup.py config"), so you may not need to run autoconf/configure at all. In one of my software projects, I was able to get rid of the autoconf/configure stuff altogether by using the corresponding distutils routines. These are much easier to use than autoconf/configure, and also guarantees that you are using the same compiler and compiler options for the config stage and the build stage.
this is another big problem of distutils. how can I tell distutils to *not* use compiler flags from python build but my own? for example, i use assert in devel builds and i don't want NDEBUG to be defined but i get it from a default python 2.3 build (on debian). i know i can rebuild a whole python and then use that for my devel builds but is not handy at all.
For some examples of "python setup.py config" (including finding external libraries and such), see
i'll check the examples. thank you very much.
federico
Federico Di Gregorio wrote:
Il lun, 2003-08-04 alle 10:11, Michiel Jan Laurens de Hoon ha scritto:
Most of the configuration machinery is already available in distutils (via "python setup.py config"), so you may not need to run autoconf/configure at all. In one of my software projects, I was able to get rid of the autoconf/configure stuff altogether by using the corresponding distutils routines. These are much easier to use than autoconf/configure, and also guarantees that you are using the same compiler and compiler options for the config stage and the build stage.
this is another big problem of distutils. how can I tell distutils to *not* use compiler flags from python build but my own? for example, i use assert in devel builds and i don't want NDEBUG to be defined but i get it from a default python 2.3 build (on debian). i know i can rebuild a whole python and then use that for my devel builds but is not handy at all.
It should be possible to undefine NDEBUG using the CFLAGS environment variable. Have a look at distutils/sysconfig.py: customize_compiler().
On Monday, Aug 4, 2003, at 08:47 Europe/Amsterdam, Brandon Long wrote:
I mailed this to Greg, and he said to try here:
I've got a project with a python wrapper/extension module (amoung others), called ClearSilver http://www.clearsilver.net/
The package uses autoconf/configure to determine which programs are available, and how to build things, etc. This includes external libaries and locations for them, like -lz, -ldb, etc.
Until recently, I've been just providing a Makefile for compiling the python extension, instead of using distutils. I don't really want to deal with compiling the shared library on all of the platforms, though, so I figured I'd go and create the setup.py script.
The trouble is, how to pass the information from configure to the setup.py. I was just going to pass them on the command line, ie from Makefile:
$(PYTHON) setup.py ext_build $(INCLUDES) $(LIBDIRS) $(LIBS)
where $(INCLUDES) is a list of -Ipaths and $(LIBDIRS) is a list of -Lpaths and $(LIBS) is a list of -llibraries.
I would also like the ability to change definitions from the command line. My use case is a bit different, though: Package Manager installs distutils-based packages automatically, and sometimes it "knows" that some of the variables that distutils grasps from lib/python2.3/config need to be modified.
I think a solution that would fit both my needs and Brandon's would be something like $(PYTHON) setup.py build INCLUDES="$(INCLUDES)" LIBDIR="$(LIBDIRS)" LIBS="$(LIBS)"
It is open to discussion whether the name=value assignments go before or after the "build" command name. The structure of distutils commands would suggest before, but we could also pick them all up in a first pass over argv (similarly to what make does, where you can put them anywhere). -- Jack Jansen, Jack.Jansen@cwi.nl, http://www.cwi.nl/~jack If I can't dance I don't want to be part of your revolution -- Emma Goldman
-
Brandon Long -
Federico Di Gregorio -
Jack Jansen -
M.-A. Lemburg -
Michiel Jan Laurens de Hoon