[Numpy-discussion] f2py: sharing F90 module data between modules

[Garry Willgoose]

Thanks for that. The docs suggest library dl is Unix only. Does that  
mean this solution will not work on Windows? Windows is on my  
implementation roadmap but I'm not quite there yet to test it.

I guess I am now thinking maybe I can assemble (using f2py) an  
(aggregated) shared library on the fly from the individual module  
shared libraries when I open my modelling environment. I could check  
the aggregated library mod dates against all the .so files of the  
components and only rebuild the aggregated library if there was a  
newer component than the aggregated library. That appears to work  
(and is fast) except for one behaviour of f2py. If I give f2py a list  
of files that are ALL .so (i.e. no fortran files) then f2py quits  
without actually doing anything, even if all the component shared  
libraries all have perfectly fine pythons interfaces from previous  
f2py builds. I can give it a trivial fortran file (module .. end  
module) and it works fine.

f2py -c --fcompiler=g95 --verbose -m stuff -L/Library/Frameworks/ 
Python.framework/Versions/Current/lib/python2.5/config -lpython2.5   
aread8.so fglobal_data.so               # doesn't work

f2py -c --fcompiler=g95 --verbose -m stuff -L/Library/Frameworks/ 
Python.framework/Versions/Current/lib/python2.5/config -lpython2.5   
aread8.so fglobal_data.so   junk.f90            # does work

Why is that problem? I can envisage a user that just wants to use the  
environment without writing any additional fortran modules (and thus  
may not even have an installed fortran compiler) and if they screw up  
mod dates on the files (by say a ftp from one machine to another ...  
for instance on our cluster the compiler is only installed on one  
machine and only binaries are moved around the cluster) then the  
environment might want to reassemble (with f2py) the aggregated  
library because it (erroneously) thinks there is a newer component  
shared library. This will fail because f2py quits when asked to  
process ONLY .so files. If I have a trivial fortran file to force  
f2py then this forces users to have a fortran compiler on their  
machine, even if they do not want to actually compile a new fortran  
module component, simply because f2py will not operate unless it is  
offered at least one fortran file.

Does this make sense or am I just being thick about this? Is there a  
way of making f2py merge a number of existing shared libraries into a  
single library without having to compile any fortran. I guess I could  
just invoke the linker directly in the case where there are no  
fortran files to compile but is nice being able to use distutils to  
get away from platform dependencies.

>> according to which makes your goal unachivable because of how
>> Python loads shared libraries *by default*, see below.
>
>> Try to use sys.setdlopenflags(...) before importing f2py generated
>> extension modules and then reset the state using sys.setdlopenflags 
>> (0).
>
> I also had to do something similar for solving a different problem,
> feel free to reuse the code here. This way, you have chances to make
> it working in a many platforms. You can put this in a __init__.py, and
> next import all your extensions inside the last try/finally block.
>
> http://projects.scipy.org/mpi4py/browser/mpi4py/trunk/src/_rtld.py
>
>
>
>
> -- 
> Lisandro Dalcín
> ---------------
> Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
> Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
> Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
> PTLC - Güemes 3450, (3000) Santa Fe, Argentina
> Tel/Fax: +54-(0)342-451.1594
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion

====================================================================
Prof Garry Willgoose,
Australian Professorial Fellow in Environmental Engineering,
Director, Centre for Climate Impact Management (C2IM),
School of Engineering, The University of Newcastle,
Callaghan, 2308
Australia.

Centre webpage: www.c2im.org.au

Phone: (International) +61 2 4921 6050 (Tues-Fri AM); +61 2 6545 9574  
(Fri PM-Mon)
FAX: (International) +61 2 4921 6991 (Uni); +61 2 6545 9574 (personal  
and Telluric)
Env. Engg. Secretary: (International) +61 2 4921 6042

email:  garry.willgoose at newcastle.edu.au;  
g.willgoose at telluricresearch.com
email-for-life: garry.willgoose at alum.mit.edu
personal webpage: www.telluricresearch.com/garry
====================================================================
"Do not go where the path may lead, go instead where there is no path  
and leave a trail"
                           Ralph Waldo Emerson
====================================================================