Numerical Python and LAPACK on 64-bit machines
Dear Numerical Python user and developers I ran into the following problem: The python application I'm developing uses Numerical Python and other C modules that call LAPACK. My application runs well on 32bit architectures: When I tried to run the application on a HP-UX 64bit machine the application produced bus errors. After a long debugging session I found out that Fortran integers are still 32bit wide on this machine. Therefore also the HP LAPACK library has to called using 32bit integers. Numerical Python however codes Fortran integers as C 'long int' variables, which are 64bit wide on this machine. To make my application run on the HP-UX 64bit machine, I had to change all 'long int' to 'int' variables in Src/lapack_litemodule.c, which is a rather painful hack (see end of message for an example). My question is: Should Fortran integers not be coded as 'int' instead of 'long int' in Numerical Python? This way, it would still work on all 32 bit machines and also on the 64-bit machines I know. Would this work on all 64-bit machines? Thanks for your comments/help. -- Roman Geus E.g. the lapack_lite_dgetrf() function now looks like this: static PyObject *lapack_lite_dgetrf(PyObject *self, PyObject *args) { int lapack_lite_status__; int m; int n; PyObject *a; int lda; PyObject *ipiv; int info; int i; int *ipiv_int; int ipiv_len; TRY(PyArg_ParseTuple(args,"iiOiOi",&m,&n,&a,&lda,&ipiv,&info)); TRY(lapack_lite_CheckObject(a,PyArray_DOUBLE,"a","PyArray_DOUBLE","dgetrf")); TRY(lapack_lite_CheckObject(ipiv,PyArray_LONG,"ipiv","PyArray_LONG","dgetrf")); ipiv_len = m < n ? m : n; ipiv_int = (int *)malloc(ipiv_len * sizeof(int)); assert(ipiv_int); for (i = 0; i < ipiv_len; i ++) ipiv_int[i] = LDATA(ipiv)[i]; #if defined(NO_APPEND_FORTRAN) lapack_lite_status__ = dgetrf(&m,&n,DDATA(a),&lda,ipiv_int,&info); #else lapack_lite_status__ = dgetrf_(&m,&n,DDATA(a),&lda,ipiv_int,&info); #endif for (i = 0; i < ipiv_len; i ++) LDATA(ipiv)[i] = ipiv_int[i]; free(ipiv); return Py_BuildValue("{s:l,s:l,s:l,s:l,s:l}","dgetrf_",(long)lapack_lite_status__,"m",(long)m,"n",(long)n,"lda",(long)lda,"info",(long)info); }
On Wednesday 06 March 2002 09:50 am, Roman Geus wrote:
Dear Numerical Python user and developers
I ran into the following problem:
The python application I'm developing uses Numerical Python and other C modules that call LAPACK. My application runs well on 32bit architectures:
When I tried to run the application on a HP-UX 64bit machine the application produced bus errors. After a long debugging session I found out that Fortran integers are still 32bit wide on this machine. Therefore also the HP LAPACK library has to called using 32bit integers. Numerical Python however codes Fortran integers as C 'long int' variables, which are 64bit wide on this machine.
Have you tried the +i8 flag for the HP fortran compiler? It converts all fortran integers to 8bytes entities. Regards, Roy.
Roy Dragseth wrote:
On Wednesday 06 March 2002 09:50 am, Roman Geus wrote:
Dear Numerical Python user and developers
I ran into the following problem:
The python application I'm developing uses Numerical Python and other C modules that call LAPACK. My application runs well on 32bit architectures:
When I tried to run the application on a HP-UX 64bit machine the application produced bus errors. After a long debugging session I found out that Fortran integers are still 32bit wide on this machine. Therefore also the HP LAPACK library has to called using 32bit integers. Numerical Python however codes Fortran integers as C 'long int' variables, which are 64bit wide on this machine.
Have you tried the +i8 flag for the HP fortran compiler? It converts all fortran integers to 8bytes entities.
Regards, Roy.
I think this wouldn't help. The optimized BLAS/LAPACK library supplied by HP expects 32bit integers and other software incorporated into my python application (e.g. SuperLU) calls BLAS/LAPACK library using 32bit integers (C 'int' type). So, what really needs to be changed (at least for this machine) is how Numerical Python calls BLAS/LAPACK. It also needs to use 32bit integers. So this means using 'int' instead of 'long int'. Regards, Roman
Hi, On Wed, 6 Mar 2002, Roman Geus wrote:
So, what really needs to be changed (at least for this machine) is how Numerical Python calls BLAS/LAPACK. It also needs to use 32bit integers. So this means using 'int' instead of 'long int'.
Having wrapped a lot of Fortran codes to Python, I agree, that Numerical Python should use 'int', instead of, 'long'. Though I have little influence to make this change to happen in Numeric but just agreeing with you. Pearu
Pearu Peterson <pearu@cens.ioc.ee> writes:
Having wrapped a lot of Fortran codes to Python, I agree, that Numerical Python should use 'int', instead of, 'long'. Though I have little influence to make this change to happen in Numeric but just agreeing with you.
Even without the Fortran aspect, I'd prefer 'int' for integer arrays in general. There may be applications that need 64-bit integers, but any portable application wouldn't rely on them anyway. 64-bit arrays take up more memory, and when you pickle them you cannot read those files on 32-bit machines. Konrad. -- ------------------------------------------------------------------------------- Konrad Hinsen | E-Mail: hinsen@cnrs-orleans.fr Centre de Biophysique Moleculaire (CNRS) | Tel.: +33-2.38.25.56.24 Rue Charles Sadron | Fax: +33-2.38.63.15.17 45071 Orleans Cedex 2 | Deutsch/Esperanto/English/ France | Nederlands/Francais -------------------------------------------------------------------------------
Hello, Pearu Peterson wrote:
Hi,
On Wed, 6 Mar 2002, Roman Geus wrote:
So, what really needs to be changed (at least for this machine) is how Numerical Python calls BLAS/LAPACK. It also needs to use 32bit integers. So this means using 'int' instead of 'long int'.
Having wrapped a lot of Fortran codes to Python, I agree, that Numerical Python should use 'int' instead of, 'long'. Though I have little influence to make this change to happen in Numeric but just agreeing with you.
Pearu
What would be the best way to convince the NumPy developers to use 'int' instead 'long' for Fortran integers? I would be willing to help making the necessary changes. -- Roman
If someone is going to make the change they should change the source to use FortranInt or some similar typedef so that one ifdef could be used to change it. I believe the current lapack/blas were made by an automatic conversion tool. It is easy to make a case that they shouldn't even be in the distribution, that rather a user should install their own library. However, this is a problem on Windows, where many users do not have a development environment, and in general, because it makes the instructions for installing more complicated. So we have sort of felt stuck with it. I have no real way of convincing myself that the proposed change won't break some other platform, although it seems unlikely. -----Original Message----- From: numpy-discussion-admin@lists.sourceforge.net [mailto:numpy-discussion-admin@lists.sourceforge.net] On Behalf Of Roman Geus Sent: Thursday, March 07, 2002 1:21 AM To: numpy-discussion@lists.sourceforge.net Subject: Re: [Numpy-discussion] Numerical Python and LAPACK on 64-bit machines Hello, Pearu Peterson wrote:
Hi,
On Wed, 6 Mar 2002, Roman Geus wrote:
So, what really needs to be changed (at least for this machine) is how Numerical Python calls BLAS/LAPACK. It also needs to use 32bit integers. So this means using 'int' instead of 'long int'.
Having wrapped a lot of Fortran codes to Python, I agree, that Numerical Python should use 'int' instead of, 'long'. Though I have little influence to make this change to happen in Numeric but just agreeing with you.
Pearu
What would be the best way to convince the NumPy developers to use 'int' instead 'long' for Fortran integers? I would be willing to help making the necessary changes. -- Roman _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
participants (5)
-
Konrad Hinsen -
Paul F Dubois -
Pearu Peterson -
Roman Geus -
Roy Dragseth