f2py from numpy 1.0.5 on OSX 10.4.11/QuadPPC fails with undefined symbols
Also using g95, which is fairly new I guess, but is a downstream requirement for the application. installed python 2.5 from darwin ports, installed most of rest of the python stuff from ports as well, numpy 1.0.4 failed repeatedly with 'too many options' error when processing cmdline options so installed 1.0.5 from svn. got same 'too many errors'. guessed and removed '--debug' from commandline: f2py --opt="-O3" -c -m \ fd_rrt1d --fcompiler=g95 --debug --link-lapack_opt *.f and that cured THAT problem. same version and same commandline worked fine on Linux, but now get the "Undefined symbols:" problem that I've seen posted elsewhere but not resolved (or not in a way that fixes my problem). Here's the last few lines of the command that was kicked off by: f2py --opt="-O3" -c -m fd_rrt1d --fcompiler=g95 --link-lapack_opt *.f (incidentally, this is the same final error from both the ports version (1.0.4) and the self-built one (1.0.5). It's obviously a link error, but to what lib and where to insert the -l specification?) I originally ran this with the environment LDFLAGS set but then ran it also with LDFLAGS set to "" and then UNset ie: $unset LDFLAGS the result is the same in all cases (see below). There's obviously a link error, but what are the missing libs and where are they? Would this have been caused by running the port install with LDFLAGS set? Thanks in advance. hjm the last lines of the build are: g95:f77: CQZ.f g95:f77: Umatrix1D.f g95:f77: fd_rrt1d.f g95:f77: /tmp/tmp28SqLa/src.macosx-10.3-ppc-2.5/fd_rrt1d-f2pywrappers.f /usr/local/bin/g95 -L/Users/hjm/lib \ /tmp/tmp28SqLa/tmp/tmp28SqLa/src.macosx-10.3-ppc-2.5/fd_rrt1dmodule.o\ /tmp/tmp28SqLa/tmp/tmp28SqLa/src.macosx-10.3-ppc-2.5/fortranobject.o \ /tmp/tmp28SqLa/CQZ.o /tmp/tmp28SqLa/Umatrix1D.o \ /tmp/tmp28SqLa/fd_rrt1d.o \ /tmp/tmp28SqLa/tmp/tmp28SqLa/src.macosx-10.3-ppc-2.5/fd_rrt1d-f2pywrappers.o\ -o ./fd_rrt1d.so -Wl,-framework -Wl,Accelerate ld: Undefined symbols: _PyArg_ParseTupleAndKeywords _PyCObject_AsVoidPtr _PyCObject_FromVoidPtr _PyCObject_Type _PyComplex_Type _PyDict_GetItemString _PyDict_SetItemString _PyErr_Clear _PyErr_Format _PyErr_NewException _PyErr_Occurred _PyErr_Print _PyErr_SetString _PyExc_ImportError _PyExc_MemoryError _PyExc_RuntimeError _PyExc_ValueError _PyFloat_Type _PyImport_ImportModule _PyInt_Type _PyModule_GetDict _PyNumber_Float _PyNumber_Int _PyObject_GetAttrString _PyObject_IsTrue _PyObject_SetAttrString _PyObject_Str _PySequence_Check _PySequence_GetItem _PyString_FromString _PyString_Type _PyType_IsSubtype _PyType_Type _Py_BuildValue _Py_InitModule4 __Py_NoneStruct _fprintf$LDBLStub _PyDict_DelItemString _PyDict_New _PyExc_AttributeError _PyExc_TypeError _PyMem_Free _PyObject_Type _PyString_AsString _PyString_ConcatAndDel _Py_FindMethod __PyObject_New _sprintf$LDBLStub _MAIN_ -- Harry Mangalam - Research Computing, NACS, E2148, Engineering Gateway, UC Irvine 92697 949 824 0084(o), 949 285 4487(c) -- [A Nation of Sheep breeds a Government of Wolves. Edward R. Murrow]
On Fri, Mar 28, 2008 at 10:52 AM, Harry Mangalam <harry.mangalam@uci.edu> wrote:
Here's the last few lines of the command that was kicked off by:
f2py --opt="-O3" -c -m fd_rrt1d --fcompiler=g95 --link-lapack_opt *.f
(incidentally, this is the same final error from both the ports version (1.0.4) and the self-built one (1.0.5). It's obviously a link error, but to what lib and where to insert the -l specification?)
I originally ran this with the environment LDFLAGS set but then ran it also with LDFLAGS set to "" and then UNset ie: $unset LDFLAGS
Can you triple-check that the "unset LDFLAGS" worked by using env(1)? You still seem to have a -L/Users/hjm/lib flag that is obviously not coming from the command line. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
Hi Robert, thanks very much for your help - responses inline below. On Friday 28 March 2008, Robert Kern wrote:
Can you triple-check that the "unset LDFLAGS" worked by using env(1)? You still seem to have a -L/Users/hjm/lib flag that is obviously not coming from the command line.
$ env |grep LDFLAGS <nothing> That LDFLAGS was coming from my .profile and I've commented it out. I just re-installed numpy (successfully, it seems) and re-tried the f2py command: f2py --opt="-O3" -c -m fd_rrt1d --fcompiler=g95 --link-lapack_opt *.f which results in an otherwise successful run except that the last few lines are identical to the one I posted before: /usr/local/bin/g95 /tmp/tmp9hmmi5/tmp/tmp9hmmi5/src.macosx-10.3-ppc-2.5/fd_rrt1dmodule.o /tmp/tmp9hmmi5/tmp/tmp9hmmi5/src.macosx-10.3-ppc-2.5/fortranobject.o /tmp/tmp9hmmi5/CQZ.o /tmp/tmp9hmmi5/Umatrix1D.o /tmp/tmp9hmmi5/fd_rrt1d.o /tmp/tmp9hmmi5/tmp/tmp9hmmi5/src.macosx-10.3-ppc-2.5/fd_rrt1d-f2pywrappers.o -o ./fd_rrt1d.so -Wl,-framework -Wl,Accelerate ld: Undefined symbols: _PyArg_ParseTupleAndKeywords _PyCObject_AsVoidPtr _PyCObject_FromVoidPtr _PyCObject_Type _PyComplex_Type _PyDict_GetItemString _PyDict_SetItemString <many more but you get the idea> However, the LDFLAGS was set when I installed a numpy via the ports system previously- could that have 'poisoned' the install by setting some variable that prevents finding the appropriate lib? And do you know which lib is not being found? I could just insert it into the final link command. Harry -- Harry Mangalam - Research Computing, NACS, E2148, Engineering Gateway, UC Irvine 92697 949 824 0084(o), 949 285 4487(c) -- [A Nation of Sheep breeds a Government of Wolves. Edward R. Murrow]
Answering part of my own question, one missing lib is (not surprisingly) libpython2.5 (add -lpython2.5) so that the link command is: /usr/local/bin/g95 -L/opt/local/lib/ \ /tmp/tmp1r96Q9/tmp/tmp1r96Q9/src.macosx-10.3-ppc-2.5/fd_rrt1dmodule.o\ /tmp/tmp1r96Q9/tmp/tmp1r96Q9/src.macosx-10.3-ppc-2.5/fortranobject.o\ /tmp/tmp1r96Q9/CQZ.o /tmp/tmp1r96Q9/Umatrix1D.o \ /tmp/tmp1r96Q9/fd_rrt1d.o \ /tmp/tmp1r96Q9/tmp/tmp1r96Q9/src.macosx-10.3-ppc-2.5/fd_rrt1d-f2pywrappers.o\ -lpython2.5 \ -lSystemStubs \ -o ./fd_rrt1d.so -Wl,-framework -Wl,Accelerate ld: Undefined symbols: _MAIN_ You'd think that this would be added automatically, but this might be due to my previous installation of python2.5 (via the ports system) with the LDFLAGS set. In order to fix this, I have to uninstall, then reinstall, the entirety of the python 2.5 dependency tree. I'll set this to run later tonight. So the only remaining undefined symbol is _MAIN_ . ...? I don't know at what level to attack this; the main fortran routine is called 'fd_rrt1d', not 'main', but this was not a problem on Linux - it compiled and linked just fine. Any ideas? Harry On Sunday 30 March 2008, Harry Mangalam wrote:
Hi Robert, thanks very much for your help - responses inline below.
On Friday 28 March 2008, Robert Kern wrote:
Can you triple-check that the "unset LDFLAGS" worked by using env(1)? You still seem to have a -L/Users/hjm/lib flag that is obviously not coming from the command line.
$ env |grep LDFLAGS <nothing>
That LDFLAGS was coming from my .profile and I've commented it out.
I just re-installed numpy (successfully, it seems) and re-tried the f2py command:
f2py --opt="-O3" -c -m fd_rrt1d --fcompiler=g95 --link-lapack_opt *.f
which results in an otherwise successful run except that the last few lines are identical to the one I posted before:
/usr/local/bin/g95 /tmp/tmp9hmmi5/tmp/tmp9hmmi5/src.macosx-10.3-ppc-2.5/fd_rrt1dmodule .o /tmp/tmp9hmmi5/tmp/tmp9hmmi5/src.macosx-10.3-ppc-2.5/fortranobject. o /tmp/tmp9hmmi5/CQZ.o /tmp/tmp9hmmi5/Umatrix1D.o /tmp/tmp9hmmi5/fd_rrt1d.o /tmp/tmp9hmmi5/tmp/tmp9hmmi5/src.macosx-10.3-ppc-2.5/fd_rrt1d-f2pyw rappers.o -o ./fd_rrt1d.so -Wl,-framework -Wl,Accelerate ld: Undefined symbols: _PyArg_ParseTupleAndKeywords _PyCObject_AsVoidPtr _PyCObject_FromVoidPtr _PyCObject_Type _PyComplex_Type _PyDict_GetItemString _PyDict_SetItemString
<many more but you get the idea>
However, the LDFLAGS was set when I installed a numpy via the ports system previously- could that have 'poisoned' the install by setting some variable that prevents finding the appropriate lib?
And do you know which lib is not being found? I could just insert it into the final link command.
Harry
-- Harry Mangalam - Research Computing, NACS, E2148, Engineering Gateway, UC Irvine 92697 949 824 0084(o), 949 285 4487(c) -- [A Nation of Sheep breeds a Government of Wolves. Edward R. Murrow]
On Sun, Mar 30, 2008 at 8:16 PM, Harry Mangalam <harry.mangalam@uci.edu> wrote:
Answering part of my own question, one missing lib is (not surprisingly) libpython2.5 (add -lpython2.5) so that the link command is:
No, it isn't. They are "-undefined dynamic_lookup -bundle", most likely. This is a deficiency of the g95 FCompiler implementation. No one has bothered to get it to work on OS X; I'm not sure if g95 even supports these flags. They were added to gcc (and accordingly gfortran) by Apple; I don't know if the g95 guy has kept up. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Sunday 30 March 2008, Robert Kern wrote:
On Sun, Mar 30, 2008 at 8:16 PM, Harry Mangalam <harry.mangalam@uci.edu> wrote:
Answering part of my own question, one missing lib is (not surprisingly) libpython2.5 (add -lpython2.5) so that the link command is:
No, it isn't. They are "-undefined dynamic_lookup -bundle", most likely. This is a deficiency of the g95 FCompiler implementation. No one has bothered to get it to work on OS X; I'm not sure if g95 even supports these flags. They were added to gcc (and accordingly gfortran) by Apple; I don't know if the g95 guy has kept up.
Hi Robert, I don't understand the "No, it isn't." part. Adding '-lpython2.5' certainly removed that long list of undefined symbols - are you saying it really had no effect and that should be: "-undefined dynamic_lookup -bundle" That had little effect, so I think I've misunderstood you. The link command that has come closest to working is: /usr/local/bin/g95 \ -L/opt/local/lib/ \ -L/Developer/SDKs/MacOSX10.4u.sdk/usr/lib \ /tmp/tmp5Ef8uL/tmp/tmp5Ef8uL/src.macosx-10.3-ppc-2.5/fd_rrt1dmodule.o\ /tmp/tmp5Ef8uL/tmp/tmp5Ef8uL/src.macosx-10.3-ppc-2.5/fortranobject.o\ /tmp/tmp5Ef8uL/CQZ.o /tmp/tmp5Ef8uL/Umatrix1D.o\ /tmp/tmp5Ef8uL/fd_rrt1d.o\ /tmp/tmp5Ef8uL/tmp/tmp5Ef8uL/src.macosx-10.3-ppc-2.5/fd_rrt1d-f2pywrappers.o\ -lpython2.5 \ -lSystemStubs\ -llapack\ -lblas\ -o ./fd_rrt1d.so\ -Wl,-framework -Wl,Accelerate ld: Undefined symbols: _MAIN_ Altho there is a requirement for g95 to support the other platforms, I'm willing to try another free compiler - do you have a recommendation? Harry -- Harry Mangalam - Research Computing, NACS, E2148, Engineering Gateway, UC Irvine 92697 949 824 0084(o), 949 285 4487(c) -- [A Nation of Sheep breeds a Government of Wolves. Edward R. Murrow]
On Sun, Mar 30, 2008 at 9:20 PM, Harry Mangalam <harry.mangalam@uci.edu> wrote:
On Sunday 30 March 2008, Robert Kern wrote:
On Sun, Mar 30, 2008 at 8:16 PM, Harry Mangalam <harry.mangalam@uci.edu> wrote:
Answering part of my own question, one missing lib is (not surprisingly) libpython2.5 (add -lpython2.5) so that the link command is:
No, it isn't. They are "-undefined dynamic_lookup -bundle", most likely. This is a deficiency of the g95 FCompiler implementation. No one has bothered to get it to work on OS X; I'm not sure if g95 even supports these flags. They were added to gcc (and accordingly gfortran) by Apple; I don't know if the g95 guy has kept up.
Hi Robert,
I don't understand the "No, it isn't." part. Adding '-lpython2.5' certainly removed that long list of undefined symbols - are you saying it really had no effect and that should be: "-undefined dynamic_lookup -bundle"
First, you need the "-bundle" in order to tell the linker to create a .so bundle. Otherwise, it tries to make an executable and (correctly) warns you that you do not have a main() function. The "-undefined dynamic_lookup" tells the linker to ignore undefined symbols and assume that they will be found when the bundle is dynamically loaded, as is the case for all of the Python symbols when the extension module gets loaded. Adding -lpython2.5 silences those error messages, but does not actually address the underlying problem.
That had little effect, so I think I've misunderstood you.
Show me the link command that got executed and the error messages which followed.
The link command that has come closest to working is:
/usr/local/bin/g95 \ -L/opt/local/lib/ \ -L/Developer/SDKs/MacOSX10.4u.sdk/usr/lib \ /tmp/tmp5Ef8uL/tmp/tmp5Ef8uL/src.macosx-10.3-ppc-2.5/fd_rrt1dmodule.o\ /tmp/tmp5Ef8uL/tmp/tmp5Ef8uL/src.macosx-10.3-ppc-2.5/fortranobject.o\ /tmp/tmp5Ef8uL/CQZ.o /tmp/tmp5Ef8uL/Umatrix1D.o\ /tmp/tmp5Ef8uL/fd_rrt1d.o\ /tmp/tmp5Ef8uL/tmp/tmp5Ef8uL/src.macosx-10.3-ppc-2.5/fd_rrt1d-f2pywrappers.o\ -lpython2.5 \ -lSystemStubs\ -llapack\ -lblas\
-o ./fd_rrt1d.so\ -Wl,-framework -Wl,Accelerate ld: Undefined symbols: _MAIN_
Altho there is a requirement for g95 to support the other platforms, I'm willing to try another free compiler - do you have a recommendation?
gfortran. Get the binary from here: http://r.research.att.com/tools/ The MacPorts gfortran may also work, but I haven't tested it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
On Sun, Mar 30, 2008 at 9:20 PM, Harry Mangalam <harry.mangalam@uci.edu> wrote:
On Sunday 30 March 2008, Robert Kern wrote:
On Sun, Mar 30, 2008 at 8:16 PM, Harry Mangalam <harry.mangalam@uci.edu> wrote:
Answering part of my own question, one missing lib is (not surprisingly) libpython2.5 (add -lpython2.5) so that the link command is:
No, it isn't. They are "-undefined dynamic_lookup -bundle", most likely. This is a deficiency of the g95 FCompiler implementation. No one has bothered to get it to work on OS X; I'm not sure if g95 even supports these flags. They were added to gcc (and accordingly gfortran) by Apple; I don't know if the g95 guy has kept up.
Hi Robert,
I don't understand the "No, it isn't." part. Adding '-lpython2.5' certainly removed that long list of undefined symbols - are you saying it really had no effect and that should be: "-undefined dynamic_lookup -bundle"
I see that you are using MacPort's non-framework Python, so you may be right that you need "-L/opt/local/lib -lpython2.5". But you will definitely need the "-bundle" option. Just for clarification, those are flags for the linker, not f2py. Pass them in using $LDFLAGS. This is what the "$LDFLAGS overrides everything" behavior is for, incidentally; working around unsupported linkers. Unfortunately, I don't know of a way to detect a non-framework Python build, so that may continue to be unsupported. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
participants (2)
-
Harry Mangalam -
Robert Kern