[Numpy-discussion] how to compile Fortran using setup.py

[Ondrej Certik]

On Fri, Mar 11, 2011 at 12:24 AM, Dag Sverre Seljebotn
<d.s.seljebotn at astro.uio.no> wrote:
> On 03/11/2011 09:13 AM, Dag Sverre Seljebotn wrote:
>> On 03/11/2011 07:57 AM, Ondrej Certik wrote:
>>> On Thu, Mar 10, 2011 at 8:25 PM, Robert Kern<robert.kern at gmail.com>   wrote:
>>>> On Thu, Mar 10, 2011 at 19:58, Ondrej Certik<ondrej at certik.cz>   wrote:
>>>>> Hi,
>>>>>
>>>>> I spent about an hour googling and didn't figure this out. Here is my setup.py:
>>>>>
>>>>> setup(
>>>>>      name = "libqsnake",
>>>>>      cmdclass = {'build_ext': build_ext},
>>>>>      version = "0.1",
>>>>>      packages = [
>>>>>          'qsnake',
>>>>>          'qsnake.calculators',
>>>>>          'qsnake.calculators.tests',
>>>>>          'qsnake.data',
>>>>>          'qsnake.mesh2d',
>>>>>          'qsnake.tests',
>>>>>          ],
>>>>>      package_data = {
>>>>>          'qsnake.tests': ['phaml_data/domain.*'],
>>>>>          },
>>>>>      include_dirs=[numpy.get_include()],
>>>>>      ext_modules = [Extension("qsnake.cmesh", [
>>>>>          "qsnake/cmesh.pyx",
>>>>>          "qsnake/fmesh.f90",
>>>>>          ])],
>>>>>      description = "Qsnake standard library",
>>>>>      license = "BSD",
>>>>> )
>>>>>
>>>>> The qsnake.cmesh extension needs to compile .pyx into .c and later to
>>>>> .o, it needs to use gfortran to compile fmesh.f90 to fmesh.o, and then
>>>>> link both things. That's it. In other words, this is what I want
>>>>> distutils to do:
>>>>>
>>>>> $ cython cmesh.pyx
>>>>> $ gcc -fPIC -o cmesh.o -c cmesh.c -I$SPKG_LOCAL/include/python2.6
>>>>> -I$SPKG_LOCAL/lib/python2.6/site-packages/numpy/core/include
>>>>> $ gfortran -fPIC -o fmesh.o -c fmesh.f90
>>>>> $ gcc -shared -o cmesh.so cmesh.o fmesh.o
>>>> Difficult if sticking with distutils of any flavor. You would have
>>>> probably to replace the build_ext command to handle both the Fortran
>>>> and the Cython. You may want to give David Cournapeau's Bento a try:
>>>>
>>>>    http://pypi.python.org/pypi/bento/
>>> Thanks Robert. I burned most of my today's evening on this, trying to
>>> replace the command, but so far I didn't figure it out. It is indeed
>>> difficult, but I wasn't sure, whether it is because I am not so
>>> familiar with distutils.
>> Kurt Smith spent much more than an evening on Fortran+distutils for the
>> first release of Fwrap, I believe you could find something in the Fwrap
>> 0.1 release...
>>
>> BUT, for Fwrap trunk, Kurt ditched all of that in favour of waf. And it
>> works much, much better (I can simply export "FC=ifort" and it finds the
>> right flags for Intel Fortran). I guess in your case, that means
>> sticking with cmake.
>>
>>> I looked at bento, but I'll simply stick to cmake. I thought that for
>>> a Python package (that uses cython + C or fortran, which I thought is
>>> a pretty standard configuration for scientific computing), I would use
>>> distutils, just like other Python packages.
>> The point of Bento as I understand it is exactly to seperate packaging
>> from building, so that it becomes possible to ditch distutils but still
>> have things behave like a Python package more or less.
>>
>> In theory at least you should be able to plug cmake into Bento (perhaps
>> by having Bento generate files included into your cmake scripts, and
>> then launching a cmake process).
>>
>> David is currently working on using Bento and waf together for building
>> NumPy.
>>
>>> With cmake, I have already figured out how to mix fotran, c, c++,
>>> Cython and Python and everything works nicely together, very robustly.
>>> So if I have to hack distutils anyway, I would write a simple
>>> distutils ->   cmake bridge.
>> Please, write a bento+cmake bridge instead :-) Too much time that could
>> have been spent better has gone into distutils already, it's time to
>> just stop using it.
>
> Also, bento comes with a distutils bridge now. So with bento->cmake, one
> should be able to go distutils->bento->cmake.

Thanks Dag for the explanation about bento. Indeed, you are right,
what I want is a solid build system, that can do the job (let's say
cmake in my case), and also create Pythonic packaging, so that it does
what people expect from a Python package.

So the right approach in my case is to stick to distutils to support
setup.py, use bento to hook things in, and write a simple bento->cmake
bridge if I have time.

Ondrej