[SciPy-user] OS X build problems with non-Apple python -- diagnosis and solution

[Zachary Pincus]

Hello folks,

I've run into some trouble building scipy for OS X. Fortunately, I've  
got it mostly licked, so I thought I'd explain how.

The issues were two:
(1) A problem with g77 and the -bundle switch. This switch must not be  
used at the beginning of the command line.
(2) The linker not being pointed at the correct set of python libraries  
for non-Apple python installs.

Here are the details:

Note that I'm following the Christopher Fonnesbeck's directions  
(http://www.scipy.org/documentation/Members/fonnesbeck/osx_build.txt )  
exactly, except that I've used Fink to install python. All other  
packages were manually installed. I'm using a CVS checkout of SciPy  
from today.

The first problem that I encountered was in building the various  
fortran libs with g77. (Full error text below as #1). The gist of the  
error is that g77 reports "couldn't run  
`/usr/local/bin/undle-gcc-3.4.2': No such file or directory". This  
problem turns out to be due to the (insane) fact that g77 has a "-b  
MACHINE" switch that is only honored if it is the first element on the  
command line. So 'g77 -bundle' makes the compiler look for the  
executable undle-gcc-3.4.2. The solution is to put something before the  
'-bundle' switch. I used -WALL because it's relatively harmless. (See:  
http://www.mail-archive.com/fink-devel@lists.sourceforge.net/ 
msg10604.html )

So it's important to make sure that -bundle is not the first switch. I  
suspect that this problem has not cropped up before because users  
without Fink-installed python will get a '-framework' switch before the  
-bundle switch. However, this might change in the future; as such it  
would be a good idea to explicitly make sure that -bundle is never  
first in the g77 command.

The second problem that I ran into was precisely the same issue that  
Christopher reported last year  
(http://www.scipy.net/pipermail/scipy-user/2004-February/002576.html ).  
That is, the linker was not finding the python libraries to link to the  
fortran libraries. The solution proffered at that time  
(http://www.scipy.net/pipermail/scipy-user/2004-February/002587.html )  
was to add specific switches to the compiler options for darwin.  
Unfortunately, these switches are specific for the apple-installed  
python framework. If a user has manually installed a new python (or has  
had fink do so for them), this fails. (see Error #2 below) I believe  
the python distutils provide a unified method to find out the correct  
libraries to link to, in order to avoid these sort of problems.

Anyhow, I don't know how to modify the distutils stuff to fix this  
automatically. I've succeeded in building SciPy only by manually fixing  
each breaking call to g77 -- a painful process indeed. Sorry I can't  
give a specific patch for the problem.

Zach Pincus
Department of Biochemistry and Program in Biomedical Informatics
Stanford University School of Medicine


Error #1
--------
gcc: build/src/Lib/fftpack/_fftpackmodule.c
/usr/local/bin/g77 -bundle  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/build/src/Lib/fftpack/ 
_fftpackmodule.o  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/Lib/fftpack/src/zfft.o  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/Lib/fftpack/src/drfft.o  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/Lib/fftpack/src/zrfft.o  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/Lib/fftpack/src/zfftnd.o  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/build/src/fortranobject.o  
-L/sw/lib -L/usr/local/lib/gcc/powerpc-apple-darwin6.8/3.4.2  
-Lbuild/temp.darwin-7.8.0-PowerMacintosh-2.3 -ldfftpack -ldrfftw  
-ldfftw -lg2c -lcc_dynamic -o  
build/lib.darwin-7.8.0-PowerMacintosh-2.3/scipy/fftpack/_fftpack.so
g77: couldn't run `/usr/local/bin/undle-gcc-3.4.2': No such file or  
directory
g77: couldn't run `/usr/local/bin/undle-gcc-3.4.2': No such file or  
directory


Error #2
--------
/usr/local/bin/g77 -WALL -bundle  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/build/src/Lib/fftpack/ 
_fftpackmodule.o  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/Lib/fftpack/src/zfft.o  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/Lib/fftpack/src/drfft.o  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/Lib/fftpack/src/zrfft.o  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/Lib/fftpack/src/zfftnd.o  
build/temp.darwin-7.8.0-PowerMacintosh-2.3/build/src/fortranobject.o  
-L/sw/lib -L/usr/local/lib/gcc/powerpc-apple-darwin6.8/3.4.2  
-Lbuild/temp.darwin-7.8.0-PowerMacintosh-2.3 -ldfftpack -ldrfftw  
-ldfftw -lg2c -lcc_dynamic -o  
build/lib.darwin-7.8.0-PowerMacintosh-2.3/scipy/fftpack/_fftpack.so
/usr/bin/ld: Undefined symbols:
_PyArg_ParseTuple
_PyArg_ParseTupleAndKeywords
_PyCObject_AsVoidPtr
_PyCObject_Type
_PyComplex_Type
_PyDict_GetItemString
_PyDict_SetItemString
_PyErr_Clear
_PyErr_NewException
_PyErr_Occurred
_PyErr_Print
_PyErr_SetString
_PyImport_ImportModule
_PyInt_Type
_PyModule_GetDict
_PyNumber_Int
_PyObject_GetAttrString
_PySequence_Check
_PySequence_GetItem
_PyString_FromString
_PyString_Type
_PyType_IsSubtype
_PyType_Type
_Py_BuildValue
_Py_FatalError
_Py_InitModule4
__Py_NoneStruct
_PyCObject_FromVoidPtr
_PyDict_DelItemString
_PyDict_New
_PyErr_Format
_PyExc_AttributeError
_PyExc_RuntimeError
_PyExc_TypeError
_PyObject_Free
_PyString_ConcatAndDel
_Py_FindMethod
__PyObject_New
collect2: ld returned 1 exit status