[Numpy-discussion] Help using numpy.distutils.fcompiler for my GSoC project

[David Cournapeau]

Kurt Smith wrote:
> On Wed, Jun 24, 2009 at 9:05 AM, David
> Cournapeau<david at ar.media.kyoto-u.ac.jp> wrote:
>   
>> Kurt Smith wrote:
>>     
>>> On Tue, Jun 23, 2009 at 10:17 PM, David Cournapeau<cournape at gmail.com> wrote:
>>>
>>>
>>>       
>>>> If possible, you should not build executables, it is not portable.
>>>> Compiling and linking is Ok, running is not. For a tool which is aimed
>>>> a general use, I think this is important. Knowing the exact tests
>>>> needed by the OP would help me to give more detailed advices.
>>>>
>>>>         
>>> Hmmm.  Thanks for the input.
>>>
>>> Ironically, the reason we're building the executable is for
>>> portability of the interoperable types.  By running the genconfig
>>> program it guarantees that we get the correct C type <-> Fortran type
>>> correspondence set up.  This is especially desirable given that
>>> compiler flags can change the size of some datatypes, which would be
>>> captured correctly by the genconfig program's output -- if everything
>>> goes as planned ;-)  We'd like to make it so that any fortran
>>> procedure can be wrapped without having to modify the kind type
>>> parameters of the arguments.
>>>
>>>       
>> Can't you do this without running executables ? What is not portable is
>> to run executables (because they cannot always run - for example cross
>> compilation on windows). Windows causes quite a headache with recent
>> version of pythons w.r.t running executables if you need to link against
>> the C runtime.
>>     
>
> What we're attempting to do is similar to Cython.  Cython is run with
> a .pyx file as argument, and outputs a standalone C file that can be
> shipped and later compiled without any Cython dependence from then
> onwards.
>
> We'd like to have f2cy do the same:  it is run with fortran source
> files as arguments, generates wrappers & various build scripts for
> convenience (a Makefile, a setup.py, SConstruct file, etc) that are
> shipped.  Later these build files/scripts are used to create the
> extension module, without any dependence on f2cy.  So these wrappers &
> build scripts must have a way to portably generate the Fortran type
> <-> C type mappings, portable between compiler and platform.
>
> A simple example for illustration:
>
> a fortran subroutine to be wrapped:
>
> subroutine foo(a)
> integer(kind=selected_int_kind(10)), intent(inout) :: a
> ...
> end subroutine foo
>
> The 'selected_int_kind(10)' call might correspond to an actual kind
> type parameter of 1,2,4 or 8 depending on the fortran compiler -- some
> compilers have the ktp correspond to the byte size, others just label
> them sequentially, so no consistent meaning is assignable to the value
> of the selected_int_kind(10) call.  It is impossible beforehand to
> know if that kind-type-parameter corresponds to a c_int, a c_long,
> etc, since the correspondence can vary platform to platform and can be
> changed with compiler flags.  Some compilers will raise an error if an
> incorrect assumption is made.
>
> We have to have the genconfig executable run beforehand to ensure that
> the right correspondence is set-up, adapted to that platform and that
> compiler.  Again, since f2cy isn't around to make sure things are
> set-up correctly, then we're stuck with running the executable to
> handle all possible points in the [compiler] x [platform] space.
>
> There may be a via media: we could have the user supply the
> type-mapping information to f2cy, which would generate the wrappers
> with these mappings assumed.  It wouldn't be as portable or general,
> but that would be the tradeoff.  It would avoid running the executable
> on those platforms that won't allow it.
>
>   
>>> This would need to be run before compiling the extension module.  Is
>>> it possible to make this portable?
>>>
>>>       
>> No. But most of the time, you can test things without running anything.
>> For example, all the type sizeofs are detected by compilation-only with
>> numpy, using some C hackery. What are the exact tests you need to do ?
>>     
>
> I assume the type sizeofs are available for the C types

The configure checks are specific to C, but maybe they can be adapted
for fortran (I don't know enough fortran to be sure, though). The code
can be found here:

http://projects.scipy.org/numpy/browser/trunk/numpy/distutils/command/config.py
(line 169)

The idea is simple: you use sizeof(type) + a shift as an index of an
array, and rely on the compiler to fail if the index is negative.

cheers,

David