
Hi All, I sent a couple of messages to f2py mailing list, but it seems like my problem has no simple solution so I thought to ask for some suggestions here. Basically, I read some huge unformatted binary files which contain time-step data from a reservoir simulation. I don't know the dimensions (i.e., lengths) of the vectors I am going to read, and I find out this information only when I start reading the file. So, I thought it would be nice to do something like: 1) Declare outputVector as allocatable; 2) Start reading the file; 3) Find the outputVector dimension and allocate it; 4) Read the data in the outputVector; 5) Return this outputVector. It works when I compile it and build it in Fortran as an executable (defining a "main" program in my f90 module), but it bombs when I try to use it from Python with the error: C:\Documents and Settings\gavana\Desktop\ECLIPSEReader>prova.py Traceback (most recent call last): File "C:\Documents and Settings\gavana\Desktop\ECLIPSEReader\prova.py", line 3, in <module> inteHead, propertyNames, propertyTypes, propertyNumbers = ECLIPSEReader.init.readinspec("OPT_INJ.INSPEC") ValueError: failed to create intent(cache|hide)|optional array-- must have defined dimensions but got (-1,) So, I have tried with a suggestion given in the f2py mailing list, and I found out that this routine works: MODULE DUMMY IMPLICIT NONE ! Ok, so I want an allocatable array as output real(8), allocatable :: realOutput(:) CONTAINS subroutine AllocateDummy(dummyInput) implicit none save ! dummyInput is *not* used, it's here just as ! an example integer, intent(in) :: dummyInput ! Allocate and build the output array allocate(realOutput(10)) realOutput(1:10) = 0.0 realOutput(3) = 3.0 realOutput(7) = 7.0 deallocate(realOutput) return end subroutine AllocateDummy END MODULE DUMMY But this one doesn't work: MODULE DUMMY IMPLICIT NONE ! Ok, so I want an allocatable array as output real(8), allocatable :: realOutput(:) integer, allocatable :: inteOutput(:) CONTAINS subroutine AllocateDummy(dummyInput) implicit none save ! dummyInput is *not* used, it's here just as ! an example integer, intent(in) :: dummyInput ! Allocate and build the output array allocate(realOutput(10)) allocate(inteOutput(20)) realOutput(1:10) = 0.0 realOutput(3) = 3.0 realOutput(7) = 7.0 inteOutput(10) = 2 deallocate(realOutput) deallocate(inteOutput) return end subroutine AllocateDummy END MODULE DUMMY The difference between the 2 scripts, is just that in the second one I want 2 allocatable arrays instead of 1. When I compile it with f2py, I get this warning from getarrdims: Building modules... Building module "dummy"... Constructing F90 module support for "dummy"... Variables: realoutput inteoutput getarrdims:warning: assumed shape array, using 0 instead of ':' getarrdims:warning: assumed shape array, using 0 instead of ':' Constructing wrapper function "dummy.allocatedummy"... allocatedummy(dummyinput) Which is not present if I compile script number 1. Actually, if I run script 2, I can't access anymore the 2 variables realoutput and inteoutput (they are not there), while with script 1 I can easily access realoutput by writing dummy.dummy.realoutput. I can't actually see any big difference between the 2 scripts... am I missing something? This is Windows XP, Python 2.5, numpy 1.0.3.1, Compaq Visual Fortran 6.6, MS Visual Studio .NET 2003. Thank you for all your suggestions, I am at loss. Andrea. "Imagination Is The Only Weapon In The War Against Reality." http://xoomer.alice.it/infinity77/

Sorry if I'm making noise, my knowledge of fortran is really little, but in your routine AllocateDummy your are fist allocating and next deallocating the arrays. Are you sure you can then access the contents of your arrays after deallocating them? How much complicated is your binary format? For simple formats, you can just use numpy to read binary data, I use this sometimes, but again, for simple formats. On 2/1/08, Andrea Gavana <andrea.gavana@gmail.com> wrote:
Hi All,
I sent a couple of messages to f2py mailing list, but it seems like my problem has no simple solution so I thought to ask for some suggestions here.
Basically, I read some huge unformatted binary files which contain time-step data from a reservoir simulation. I don't know the dimensions (i.e., lengths) of the vectors I am going to read, and I find out this information only when I start reading the file. So, I thought it would be nice to do something like:
1) Declare outputVector as allocatable; 2) Start reading the file; 3) Find the outputVector dimension and allocate it; 4) Read the data in the outputVector; 5) Return this outputVector.
It works when I compile it and build it in Fortran as an executable (defining a "main" program in my f90 module), but it bombs when I try to use it from Python with the error:
C:\Documents and Settings\gavana\Desktop\ECLIPSEReader>prova.py Traceback (most recent call last): File "C:\Documents and Settings\gavana\Desktop\ECLIPSEReader\prova.py", line 3, in <module> inteHead, propertyNames, propertyTypes, propertyNumbers = ECLIPSEReader.init.readinspec("OPT_INJ.INSPEC") ValueError: failed to create intent(cache|hide)|optional array-- must have defined dimensions but got (-1,)
So, I have tried with a suggestion given in the f2py mailing list, and I found out that this routine works:
MODULE DUMMY IMPLICIT NONE
! Ok, so I want an allocatable array as output
real(8), allocatable :: realOutput(:)
CONTAINS
subroutine AllocateDummy(dummyInput)
implicit none save
! dummyInput is *not* used, it's here just as ! an example integer, intent(in) :: dummyInput
! Allocate and build the output array allocate(realOutput(10))
realOutput(1:10) = 0.0 realOutput(3) = 3.0 realOutput(7) = 7.0
deallocate(realOutput)
return
end subroutine AllocateDummy
END MODULE DUMMY
But this one doesn't work:
MODULE DUMMY IMPLICIT NONE
! Ok, so I want an allocatable array as output
real(8), allocatable :: realOutput(:) integer, allocatable :: inteOutput(:)
CONTAINS
subroutine AllocateDummy(dummyInput)
implicit none save
! dummyInput is *not* used, it's here just as ! an example integer, intent(in) :: dummyInput
! Allocate and build the output array allocate(realOutput(10)) allocate(inteOutput(20))
realOutput(1:10) = 0.0 realOutput(3) = 3.0 realOutput(7) = 7.0
inteOutput(10) = 2
deallocate(realOutput) deallocate(inteOutput)
return
end subroutine AllocateDummy
END MODULE DUMMY
The difference between the 2 scripts, is just that in the second one I want 2 allocatable arrays instead of 1. When I compile it with f2py, I get this warning from getarrdims:
Building modules... Building module "dummy"... Constructing F90 module support for "dummy"... Variables: realoutput inteoutput getarrdims:warning: assumed shape array, using 0 instead of ':' getarrdims:warning: assumed shape array, using 0 instead of ':' Constructing wrapper function "dummy.allocatedummy"... allocatedummy(dummyinput)
Which is not present if I compile script number 1. Actually, if I run script 2, I can't access anymore the 2 variables realoutput and inteoutput (they are not there), while with script 1 I can easily access realoutput by writing dummy.dummy.realoutput. I can't actually see any big difference between the 2 scripts... am I missing something?
This is Windows XP, Python 2.5, numpy 1.0.3.1, Compaq Visual Fortran 6.6, MS Visual Studio .NET 2003.
Thank you for all your suggestions, I am at loss.
Andrea.
"Imagination Is The Only Weapon In The War Against Reality." http://xoomer.alice.it/infinity77/ _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- Lisandro Dalcín --------------- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594

Hi Lisandro, On Feb 1, 2008 1:59 PM, Lisandro Dalcin wrote:
Sorry if I'm making noise, my knowledge of fortran is really little, but in your routine AllocateDummy your are fist allocating and next deallocating the arrays. Are you sure you can then access the contents of your arrays after deallocating them?
Thank you for your answer. Unfortunately it seems that it doesn't matter whether I deallocate them or not, I still get the compilation warning and I can't access those variable in any case. It seems like f2py (or python or whatever) does not like having more than 1 allocatable array inside a MODULE declaration.
How much complicated is your binary format?
*Very* complex. The fact is, I already know how to read those files in Fortran, is the linking with Python via f2py that is driving me mad. I can't believe no one has used before allocatable arrays as outputs (whether from a subroutine or from a module).
On 2/1/08, Andrea Gavana <andrea.gavana@gmail.com> wrote:
Hi All,
I sent a couple of messages to f2py mailing list, but it seems like my problem has no simple solution so I thought to ask for some suggestions here.
Basically, I read some huge unformatted binary files which contain time-step data from a reservoir simulation. I don't know the dimensions (i.e., lengths) of the vectors I am going to read, and I find out this information only when I start reading the file. So, I thought it would be nice to do something like:
1) Declare outputVector as allocatable; 2) Start reading the file; 3) Find the outputVector dimension and allocate it; 4) Read the data in the outputVector; 5) Return this outputVector.
It works when I compile it and build it in Fortran as an executable (defining a "main" program in my f90 module), but it bombs when I try to use it from Python with the error:
C:\Documents and Settings\gavana\Desktop\ECLIPSEReader>prova.py Traceback (most recent call last): File "C:\Documents and Settings\gavana\Desktop\ECLIPSEReader\prova.py", line 3, in <module> inteHead, propertyNames, propertyTypes, propertyNumbers = ECLIPSEReader.init.readinspec("OPT_INJ.INSPEC") ValueError: failed to create intent(cache|hide)|optional array-- must have defined dimensions but got (-1,)
So, I have tried with a suggestion given in the f2py mailing list, and I found out that this routine works:
MODULE DUMMY IMPLICIT NONE
! Ok, so I want an allocatable array as output
real(8), allocatable :: realOutput(:)
CONTAINS
subroutine AllocateDummy(dummyInput)
implicit none save
! dummyInput is *not* used, it's here just as ! an example integer, intent(in) :: dummyInput
! Allocate and build the output array allocate(realOutput(10))
realOutput(1:10) = 0.0 realOutput(3) = 3.0 realOutput(7) = 7.0
deallocate(realOutput)
return
end subroutine AllocateDummy
END MODULE DUMMY
But this one doesn't work:
MODULE DUMMY IMPLICIT NONE
! Ok, so I want an allocatable array as output
real(8), allocatable :: realOutput(:) integer, allocatable :: inteOutput(:)
CONTAINS
subroutine AllocateDummy(dummyInput)
implicit none save
! dummyInput is *not* used, it's here just as ! an example integer, intent(in) :: dummyInput
! Allocate and build the output array allocate(realOutput(10)) allocate(inteOutput(20))
realOutput(1:10) = 0.0 realOutput(3) = 3.0 realOutput(7) = 7.0
inteOutput(10) = 2
deallocate(realOutput) deallocate(inteOutput)
return
end subroutine AllocateDummy
END MODULE DUMMY
The difference between the 2 scripts, is just that in the second one I want 2 allocatable arrays instead of 1. When I compile it with f2py, I get this warning from getarrdims:
Building modules... Building module "dummy"... Constructing F90 module support for "dummy"... Variables: realoutput inteoutput getarrdims:warning: assumed shape array, using 0 instead of ':' getarrdims:warning: assumed shape array, using 0 instead of ':' Constructing wrapper function "dummy.allocatedummy"... allocatedummy(dummyinput)
Which is not present if I compile script number 1. Actually, if I run script 2, I can't access anymore the 2 variables realoutput and inteoutput (they are not there), while with script 1 I can easily access realoutput by writing dummy.dummy.realoutput. I can't actually see any big difference between the 2 scripts... am I missing something?
This is Windows XP, Python 2.5, numpy 1.0.3.1, Compaq Visual Fortran 6.6, MS Visual Studio .NET 2003.
Thank you for all your suggestions, I am at loss.
Andrea.
"Imagination Is The Only Weapon In The War Against Reality." http://xoomer.alice.it/infinity77/ _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- Lisandro Dalcín --------------- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594 _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-- Andrea. "Imagination Is The Only Weapon In The War Against Reality." http://xoomer.alice.it/infinity77/

On Fri, February 1, 2008 4:18 pm, Andrea Gavana wrote:
Hi Lisandro,
On Feb 1, 2008 1:59 PM, Lisandro Dalcin wrote:
Sorry if I'm making noise, my knowledge of fortran is really little, but in your routine AllocateDummy your are fist allocating and next deallocating the arrays. Are you sure you can then access the contents of your arrays after deallocating them?
Thank you for your answer.
Unfortunately it seems that it doesn't matter whether I deallocate them or not, I still get the compilation warning and I can't access those variable in any case.
You cannot access those becase they are deallocated. Try to disable deallocate statements in your fortran code.
It seems like f2py (or python or whatever) does not like having more than 1 allocatable array inside a MODULE declaration.
This is not true.
How much complicated is your binary format?
*Very* complex. The fact is, I already know how to read those files in Fortran, is the linking with Python via f2py that is driving me mad. I can't believe no one has used before allocatable arrays as outputs (whether from a subroutine or from a module).
You can use allocatable arrays from module in Python as described in f2py users guide. It could be that the problem is related to deallocating the arrays in the fortran code. Regards, Pearu

On Fri, February 1, 2008 1:28 pm, Andrea Gavana wrote:
Hi All,
I sent a couple of messages to f2py mailing list, but it seems like my problem has no simple solution so I thought to ask for some suggestions here.
Sorry, I haven't been around there long time.
Basically, I read some huge unformatted binary files which contain time-step data from a reservoir simulation. I don't know the dimensions (i.e., lengths) of the vectors I am going to read, and I find out this information only when I start reading the file. So, I thought it would be nice to do something like:
1) Declare outputVector as allocatable; 2) Start reading the file; 3) Find the outputVector dimension and allocate it; 4) Read the data in the outputVector;
looks ok.
5) Return this outputVector.
What do you mean by "return"? You cannot return allocatable arrays as far as comes to using f2py for generating wrappers. However, you can access allocatable array outputVector if it is module data, as you do below.
It works when I compile it and build it in Fortran as an executable (defining a "main" program in my f90 module), but it bombs when I try to use it from Python with the error:
C:\Documents and Settings\gavana\Desktop\ECLIPSEReader>prova.py Traceback (most recent call last): File "C:\Documents and Settings\gavana\Desktop\ECLIPSEReader\prova.py", line 3, in <module> inteHead, propertyNames, propertyTypes, propertyNumbers = ECLIPSEReader.init.readinspec("OPT_INJ.INSPEC") ValueError: failed to create intent(cache|hide)|optional array-- must have defined dimensions but got (-1,)
This exception is not directly related to what follows below.
So, I have tried with a suggestion given in the f2py mailing list, and I found out that this routine works:
MODULE DUMMY IMPLICIT NONE
! Ok, so I want an allocatable array as output
real(8), allocatable :: realOutput(:) ...
END MODULE DUMMY
But this one doesn't work:
MODULE DUMMY IMPLICIT NONE
! Ok, so I want an allocatable array as output
real(8), allocatable :: realOutput(:) integer, allocatable :: inteOutput(:)
CONTAINS ... END MODULE DUMMY
This one works fine here: $ f2py -c -m m2 m2.f90 --fcompiler=gnu95
import m2 print m2.dummy.__doc__ realoutput - 'd'-array(-1), not allocated inteoutput - 'i'-array(-1), not allocated allocatedummy - Function signature: allocatedummy(dummyinput) Required arguments: dummyinput : input int
The difference between the 2 scripts, is just that in the second one I want 2 allocatable arrays instead of 1. When I compile it with f2py, I get this warning from getarrdims:
Building modules... Building module "dummy"... Constructing F90 module support for "dummy"... Variables: realoutput inteoutput ^^^^^^^^^^^^^^^^^^^^^
Looks like both allocatable arrays should be present also in your case.
getarrdims:warning: assumed shape array, using 0 instead of ':' getarrdims:warning: assumed shape array, using 0 instead of ':'
These warnings can be ignored.
Which is not present if I compile script number 1. Actually, if I run script 2, I can't access anymore the 2 variables realoutput and inteoutput (they are not there), while with script 1 I can easily access realoutput by writing dummy.dummy.realoutput.
What do you mean by accessing? What happens when you type: dummy.dummy.inteoutput ? Note that when you call AllocateDummy function, then you allocate and then deallocate the arrays. So, in Python dummy.dummy.realoutput and dummy.dummy.inteoutput should always return None. See http://cens.ioc.ee/projects/f2py2e/usersguide/index.html#allocatable-arrays for how to use allocatable module data from Python.
I can't actually see any big difference between the 2 scripts... am I missing something?
This is Windows XP, Python 2.5, numpy 1.0.3.1, Compaq Visual Fortran 6.6, MS Visual Studio .NET 2003.
I am using numpy from svn. Regards, Pearu

Pearu Peterson wrote:
On Fri, February 1, 2008 1:28 pm, Andrea Gavana wrote:
Hi All,
I sent a couple of messages to f2py mailing list, but it seems like my problem has no simple solution so I thought to ask for some suggestions here.
Sorry, I haven't been around there long time.
Are you going to continue not reading the f2py list? If so, you should point everyone there to this list and close the list. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

On Fri, February 1, 2008 8:39 pm, Robert Kern wrote:
Pearu Peterson wrote:
On Fri, February 1, 2008 1:28 pm, Andrea Gavana wrote:
Hi All,
I sent a couple of messages to f2py mailing list, but it seems like my problem has no simple solution so I thought to ask for some suggestions here.
Sorry, I haven't been around there long time.
Are you going to continue not reading the f2py list? If so, you should point everyone there to this list and close the list.
It is a bit embarrassing, I haven't read the list because I lost the link to f2py list archives that I used to use in my mailing box in the server.. but I have been also busy with non-python stuff last years (I moved and got married..:). Anyway, I have subscribed to the f2py list again I'll try to respond to any messages that have unresolved issues, also in the arhives. Thanks, Pearu

Pearu Peterson wrote:
On Fri, February 1, 2008 8:39 pm, Robert Kern wrote:
On Fri, February 1, 2008 1:28 pm, Andrea Gavana wrote:
Hi All,
I sent a couple of messages to f2py mailing list, but it seems like my problem has no simple solution so I thought to ask for some suggestions here. Sorry, I haven't been around there long time. Are you going to continue not reading the f2py list? If so, you should
Pearu Peterson wrote: point everyone there to this list and close the list.
It is a bit embarrassing, I haven't read the list because I lost the link to f2py list archives that I used to use in my mailing box in the server.. but I have been also busy with non-python stuff last years (I moved and got married..:). Anyway, I have subscribed to the f2py list again I'll try to respond to any messages that have unresolved issues, also in the arhives.
Great. Thank you. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

On 2/1/08, Pearu Peterson <pearu@cens.ioc.ee> wrote:
Sorry, I haven't been around there long time.
Are you going to continue not reading the f2py list? If so, you should point everyone there to this list and close the list.
Anyway, I have subscribed to the f2py list again I'll try to respond to any messages that have unresolved issues, also in the arhives.
Pearu, now that f2py is part of numpy, I think it would be easier for you and also for users to post to the numpy list for f2py-related issues. What do you think? -- Lisandro Dalcín --------------- Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC) Instituto de Desarrollo Tecnológico para la Industria Química (INTEC) Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET) PTLC - Güemes 3450, (3000) Santa Fe, Argentina Tel/Fax: +54-(0)342-451.1594

On Mon, February 4, 2008 4:39 pm, Lisandro Dalcin wrote:
Pearu, now that f2py is part of numpy, I think it would be easier for you and also for users to post to the numpy list for f2py-related issues. What do you think?
Personaly, I don't have strong opinions on this. On one hand, it would be better if f2py related issues are raised in one and only one list. It could be numpy list as f2py issues would not add much extra traffic to it. On the other hand, if redirecting f2py-users messages to numpy list and also vice versa, then I am not sure that all subscribes in f2py-users list will appreciate extra messages about numpy issues that might be irrelevant to them. Currently there are about 180 subscribes to the f2py-users list, many of them also subscribe numpy-discussion list, I guess. If most of them would subscribe numpy-discussion list then we could get rid of f2py-users list. If the administrators of the numpy-discussion list could share (privately) the list of subscribers mails with me then I could check how many of the f2py users subscribe the numpy list and it would be easier to make a decision on the future of f2py-users list. When I'll be back to working on f2py g3 then I'll probably create a google code project for it. There we'll have separate list for the future version of f2py anyway. Regards, Pearu
participants (4)
-
Andrea Gavana
-
Lisandro Dalcin
-
Pearu Peterson
-
Robert Kern