Problems building numpy and scipy on AIX
Hello, I've run into a problem building numpy-1.0.2 on AIX (using gcc and native Fortran compilers). The problem on that platform in general is that the process for building shared libraries is different from what's normally done (and it's a pain...) Anyway. Core python has a dedicated script called ld_so_aix which is used to create the shared objects (it can be found in the .../lib/python2.5/config directory of any installed python version). Everything compiled with the compiler that was used to build python itself, and which is compiled through the standard distutils, is also using this construction, so building C extensions usually works fine. I ran into problems during the installation of numpy 1.0.2 using python setup.py config_fc --fcompiler=ibm install because the numpy distutils try to use the xlf95 compiler to create a shared image - what needs to be done instead is to wrap the call to the compiler with the ld_so_aix script. I have attached a patch which solves the problem for me (and python 2.5), but I don't know if it is the right place and way to do that - and the python version is hardwired as well, so it's not really a fix. What I also find a bit suspicuous is that the Fortran linker is used in that particular case at all - the problem occurs when building _dotblas.so, and some targets after that one. However, the corresponding code is written in C and has actually been compiled with the C compiler. The problem did not appear with numpy 1.0.1, where the C compiler is (correctly) used for linking. So maybe there's another hidden problem... But anyway, I think that the missing wrapping with ld_so_aix is a bug on its own. When moving on to building scipy, I ran into similar problems with C++ compiled code - instead of wrapping the c++ compiler/linker with ld_so_aix, the following command is executed g++ gcc -pthread -bI:/homespace/grasppf/aix/lib/python2.5/config/python.exp build/temp.aix-5.1-2.5/Lib/cluster/src/vq_wrap.o -Lbuild/temp.aix-5.1-2.5 -o build/lib.aix-5.1-2.5/scipy/cluster/_vq.so and predictably causes an error. What's happening is (I think) that the numpy distutils partially overwrite the linker modifications from the core python. (the -pthread -bI:/...python.exp is an argument to the ld_so_aix script). My problem is that I do not know where in the numpy distutils code this modification happens, so I've no idea where to try to fix it - does anyone on this list know? Many thanks, Christian.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Christian Marquardt wrote:
Hello,
I've run into a problem building numpy-1.0.2 on AIX (using gcc and native Fortran compilers). The problem on that platform in general is that the process for building shared libraries is different from what's normally done (and it's a pain...)
Already fixed in svn :) - -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGJjctN9ixZKFWjRQRAoLZAJ4uz6L/dO1j47nz4o5BEFiFLlc6bwCfayha tWZCkDzXjNR7lrJK7AVMyTc= =9it9 -----END PGP SIGNATURE-----
Dear David, the svn version of numpy does indeed build cleanly on AIX. Many thanks! However, the wrapper problem still exists for the C++ compiler, and shows up when compiling scipy. Now, I *assume* that SciPy is using the distutils as installed by numpy. Do you know where the linker settings for the C++ compiler might be overwritten? There are two or three C compiler related python modules in numpy/distutils... Or would you think that this problem is entirely unrelated to the distutils in numpy? Many thanks, Christian. On Wed, April 18, 2007 17:20, David M. Cooke wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Christian Marquardt wrote:
Hello,
I've run into a problem building numpy-1.0.2 on AIX (using gcc and native Fortran compilers). The problem on that platform in general is that the process for building shared libraries is different from what's normally done (and it's a pain...)
Already fixed in svn :)
- -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFGJjctN9ixZKFWjRQRAoLZAJ4uz6L/dO1j47nz4o5BEFiFLlc6bwCfayha tWZCkDzXjNR7lrJK7AVMyTc= =9it9 -----END PGP SIGNATURE----- _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Christian Marquardt wrote:
Dear David,
the svn version of numpy does indeed build cleanly on AIX. Many thanks!
However, the wrapper problem still exists for the C++ compiler, and shows up when compiling scipy. Now, I *assume* that SciPy is using the distutils as installed by numpy. Do you know where the linker settings for the C++ compiler might be overwritten? There are two or three C compiler related python modules in numpy/distutils... Or would you think that this problem is entirely unrelated to the distutils in numpy?
I'm working on a better solution, but the quick fix to your problem is to look in numpy/distutils/command/build_ext.py. There are two lines that reference self.compiler.linker_so[0]; change those 0s to a 1s, so it keeps the ld_so_aix script there when switching the linker. - -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGJ9MzN9ixZKFWjRQRArNwAKC029wYORk9sm+FShcYKNd0UEcMdgCghHGC rjYqtaESdt8zRgZHCDxYbDk= =PS30 -----END PGP SIGNATURE-----
Yes, that worked - many thanks! Christian. On Thu, April 19, 2007 22:38, David M. Cooke wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Christian Marquardt wrote:
Dear David,
the svn version of numpy does indeed build cleanly on AIX. Many thanks!
However, the wrapper problem still exists for the C++ compiler, and shows up when compiling scipy. Now, I *assume* that SciPy is using the distutils as installed by numpy. Do you know where the linker settings for the C++ compiler might be overwritten? There are two or three C compiler related python modules in numpy/distutils... Or would you think that this problem is entirely unrelated to the distutils in numpy?
I'm working on a better solution, but the quick fix to your problem is to look in numpy/distutils/command/build_ext.py. There are two lines that reference self.compiler.linker_so[0]; change those 0s to a 1s, so it keeps the ld_so_aix script there when switching the linker.
- -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFGJ9MzN9ixZKFWjRQRArNwAKC029wYORk9sm+FShcYKNd0UEcMdgCghHGC rjYqtaESdt8zRgZHCDxYbDk= =PS30 -----END PGP SIGNATURE----- _______________________________________________ Numpy-discussion mailing list Numpy-discussion@scipy.org http://projects.scipy.org/mailman/listinfo/numpy-discussion
On Apr 21, 2007, at 09:03 , Christian Marquardt wrote:
Yes,
that worked - many thanks!
FWIW, svn should work out of the box now.
On Thu, April 19, 2007 22:38, David M. Cooke wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Christian Marquardt wrote:
Dear David,
the svn version of numpy does indeed build cleanly on AIX. Many thanks!
However, the wrapper problem still exists for the C++ compiler, and shows up when compiling scipy. Now, I *assume* that SciPy is using the distutils as installed by numpy. Do you know where the linker settings for the C++ compiler might be overwritten? There are two or three C compiler related python modules in numpy/distutils... Or would you think that this problem is entirely unrelated to the distutils in numpy?
I'm working on a better solution, but the quick fix to your problem is to look in numpy/distutils/command/build_ext.py. There are two lines that reference self.compiler.linker_so[0]; change those 0s to a 1s, so it keeps the ld_so_aix script there when switching the linker.
- -- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (Darwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iD8DBQFGJ9MzN9ixZKFWjRQRArNwAKC029wYORk9sm+FShcYKNd0UEcMdgCghHGC rjYqtaESdt8zRgZHCDxYbDk= =PS30 -----END PGP SIGNATURE-----
-- |>|\/|< /------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca
participants (2)
-
Christian Marquardt
-
David M. Cooke