[Numpy-discussion] how to compile Fortran using setup.py

[David Cournapeau]

On Fri, Mar 11, 2011 at 1:57 AM, Ondrej Certik <ondrej at certik.cz> wrote:
> On Thu, Mar 10, 2011 at 8:25 PM, Robert Kern <robert.kern at gmail.com> wrote:
>> On Thu, Mar 10, 2011 at 19:58, Ondrej Certik <ondrej at certik.cz> wrote:
>>> Hi,
>>>
>>> I spent about an hour googling and didn't figure this out. Here is my setup.py:
>>>
>>> setup(
>>>    name = "libqsnake",
>>>    cmdclass = {'build_ext': build_ext},
>>>    version = "0.1",
>>>    packages = [
>>>        'qsnake',
>>>        'qsnake.calculators',
>>>        'qsnake.calculators.tests',
>>>        'qsnake.data',
>>>        'qsnake.mesh2d',
>>>        'qsnake.tests',
>>>        ],
>>>    package_data = {
>>>        'qsnake.tests': ['phaml_data/domain.*'],
>>>        },
>>>    include_dirs=[numpy.get_include()],
>>>    ext_modules = [Extension("qsnake.cmesh", [
>>>        "qsnake/cmesh.pyx",
>>>        "qsnake/fmesh.f90",
>>>        ])],
>>>    description = "Qsnake standard library",
>>>    license = "BSD",
>>> )
>>>
>>> The qsnake.cmesh extension needs to compile .pyx into .c and later to
>>> .o, it needs to use gfortran to compile fmesh.f90 to fmesh.o, and then
>>> link both things. That's it. In other words, this is what I want
>>> distutils to do:
>>>
>>> $ cython cmesh.pyx
>>> $ gcc -fPIC -o cmesh.o -c cmesh.c -I$SPKG_LOCAL/include/python2.6
>>> -I$SPKG_LOCAL/lib/python2.6/site-packages/numpy/core/include
>>> $ gfortran -fPIC -o fmesh.o -c fmesh.f90
>>> $ gcc -shared -o cmesh.so cmesh.o fmesh.o
>>
>> Difficult if sticking with distutils of any flavor. You would have
>> probably to replace the build_ext command to handle both the Fortran
>> and the Cython. You may want to give David Cournapeau's Bento a try:
>>
>>  http://pypi.python.org/pypi/bento/
>
> Thanks Robert. I burned most of my today's evening on this, trying to
> replace the command, but so far I didn't figure it out. It is indeed
> difficult, but I wasn't sure, whether it is because I am not so
> familiar with distutils.
>
> I looked at bento, but I'll simply stick to cmake. I thought that for
> a Python package (that uses cython + C or fortran, which I thought is
> a pretty standard configuration for scientific computing), I would use
> distutils, just like other Python packages.

The initial reason why I started bento is because this is practically
impossible: distutils is very adverse to changes, and you cannot
easily change this (I know because that's exactly what numscons was).

For example, if you generate a cmake file from distutils, you will
quickly realize that getting the necessary data is not easy - if you
create a build_cmake, you still need install options, but those are
not  fully available before install is run (you can do some very
horrible hacks to get most of them by copying running commands).
Instead bento is based around the usual configure/build/install ala
autoconf, where all the options are resolved after configure.

I am not pretending that bento can solve all your issues, but you can
*today* build non trivial python packages with it, including metadata,
eggs and windows installers. And there is already a proof of concept
to use either distutils compiler classes, waf and a custom build tool
without the need to customize anything outside the build phase (i.e.
install is exactly the same and does not even need to know about
build).

As for supporting setup.py, I have started something in that direction:

http://cournape.wordpress.com/2011/03/06/adding-a-distutils-compatibility-layer-to-bento/

cheers,

David