[Numpy-discussion] Re: Problems building numpy w/ ATLAS on Solaris 8

Tue Feb 21 14:10:12 EST 2006

skip at pobox.com wrote:
> After a brief hiatus I'm back to trying to build numpy.  Last time I checked
> in (on the scipy list) I had successfully built ATLAS and created this
> simple site.cfg file in .../numpy/distutils/site.cfg:
> 
>     [atlas]
>     library_dirs = /home/titan/skipm/src/ATLAS/lib/SunOS_Babe
>     include_dirs = /home/titan/skipm/src/ATLAS/include/SunOS_Babe
>     # for overriding the names of the atlas libraries
>     atlas_libs = lapack, f77blas, cblas, atlas
> 
> I svn up'd (now at rev 2138), zapped my build directory, then executed
> "python setup.py build".  Just in case it matters, I'm using Python 2.4.2
> built with GCC 3.4.1 on Solaris 8.  Here's the output of my build attempt:
> 
>     Running from numpy source directory.
>     No module named __svn_version__
>     F2PY Version 2_2138
>     blas_opt_info:
>     blas_mkl_info:
>     /home/ink/skipm/src/numpy/numpy/distutils/system_info.py:531: UserWarning: Library error: libs=['mkl', 'vml', 'guide'] found_libs=[]
>       warnings.warn("Library error: libs=%s found_libs=%s" % \
>       NOT AVAILABLE
>     
>     atlas_blas_threads_info:
>     Setting PTATLAS=ATLAS
>     /home/ink/skipm/src/numpy/numpy/distutils/system_info.py:531: UserWarning: Library error: libs=['lapack', 'f77blas', 'cblas', 'atlas'] found_libs=[]
>       warnings.warn("Library error: libs=%s found_libs=%s" % \
>     Setting PTATLAS=ATLAS
>     Setting PTATLAS=ATLAS
>       FOUND:
>         libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
>         library_dirs = ['/home/titan/skipm/src/ATLAS/lib/SunOS_Babe']
>         language = c
>         include_dirs = ['/opt/include']
>     ...
> 
> See my site.cfg file?  Why does it affect library_dirs but not include_dirs?

Probably a bug, but I don't see exactly where at the moment. It shouldn't really
affect anything, I don't think. The only header file that comes with ATLAS is
cblas.h, and I'm pretty sure that numpy itself doesn't need it. In fact, we
provide our own copy in numpy/core/blasdot/ for the parts that can use it.

>     running build_src
>     building extension "atlas_version" sources
>       adding 'build/src/atlas_version_0x33c6fa32.c' to sources.
>     running build_ext
>     customize UnixCCompiler
>     customize UnixCCompiler using build_ext
>     building 'atlas_version' extension
>     compiling C sources
>     gcc options: '-fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC'
>     compile options: '-I/opt/include -Inumpy/core/include -I/opt/app/g++lib6/python-2.4/include/python2.4 -c'
>     /opt/lang/gcc-3.4/bin/gcc -shared build/temp.solaris-2.8-i86pc-2.4/build/src/atlas_version_0x33c6fa32.o -L/home/titan/skipm/src/ATLAS/lib/SunOS_Babe -llapack -lf77blas -lcblas -latlas -o build/temp.solaris-2.8-i86pc-2.4/atlas_version.so
>     Text relocation remains                         referenced
>         against symbol                  offset      in file
>     <unknown>                           0x7         /home/titan/skipm/src/ATLAS/lib/SunOS_Babe/libatlas.a(ATL_buildinfo.o)
>     <unknown>                           0xc         /home/titan/skipm/src/ATLAS/lib/SunOS_Babe/libatlas.a(ATL_buildinfo.o)
>     ... bunch of missing <unknown>s elided ...
>     printf                              0x1b        /home/titan/skipm/src/ATLAS/lib/SunOS_Babe/libatlas.a(ATL_buildinfo.o)
>     printf                              0x2d        /home/titan/skipm/src/ATLAS/lib/SunOS_Babe/libatlas.a(ATL_buildinfo.o)
>     printf                              0x3f        /home/titan/skipm/src/ATLAS/lib/SunOS_Babe/libatlas.a(ATL_buildinfo.o)
>     printf                              0x51        /home/titan/skipm/src/ATLAS/lib/SunOS_Babe/libatlas.a(ATL_buildinfo.o)
>     ... what's this? can't find printf??? ...
>     ld: fatal: relocations remain against allocatable but non-writable sections
>     collect2: ld returned 1 exit status

Hmm. Was ATLAS compiled -fPIC? I'm afraid I'm a little out of my depth when it
comes to linking shared objects on Solaris.

>     Text relocation remains                         referenced
>         against symbol                  offset      in file
>     <unknown>                           0x7         /home/titan/skipm/src/ATLAS/lib/SunOS_Babe/libatlas.a(ATL_buildinfo.o)
>     ... more eliding ...
>     printf                              0x108       /home/titan/skipm/src/ATLAS/lib/SunOS_Babe/libatlas.a(ATL_buildinfo.o)
>     ld: fatal: relocations remain against allocatable but non-writable sections
>     collect2: ld returned 1 exit status
>     ##### msg: error: Command "/opt/lang/gcc-3.4/bin/gcc -shared build/temp.solaris-2.8-i86pc-2.4/build/src/atlas_version_0x33c6fa32.o -L/home/titan/skipm/src/ATLAS/lib/SunOS_Babe -llapack -lf77blas -lcblas -latlas -o build/temp.solaris-2.8-i86pc-2.4/atlas_version.so" failed with exit status 1
>     error: Command "/opt/lang/gcc-3.4/bin/gcc -shared build/temp.solaris-2.8-i86pc-2.4/build/src/atlas_version_0x33c6fa32.o -L/home/titan/skipm/src/ATLAS/lib/SunOS_Babe -llapack -lf77blas -lcblas -latlas -o build/temp.solaris-2.8-i86pc-2.4/atlas_version.so" failed with exit status 1
>       FOUND:
>         libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
>         library_dirs = ['/home/titan/skipm/src/ATLAS/lib/SunOS_Babe']
>         language = c
>         define_macros = [('NO_ATLAS_INFO', 2)]
>         include_dirs = ['/opt/include']
>     
>     Warning: distutils distribution has been initialized, it may be too late to add an extension _dotblas
>     ...
> 
> How can I initialize things earlier? Does it matter?

You get messages like this when something previous goes wrong. There's nothing
you can do to initialize things earlier except to make sure that the previous
steps don't fail. It's not the most informative error message, I know.

>     Traceback (most recent call last):
>       File "setup.py", line 76, in ?
>         setup_package()
>       File "setup.py", line 63, in setup_package
>         config.add_subpackage('numpy')
>       File "/home/ink/skipm/src/numpy/numpy/distutils/misc_util.py", line 592, in add_subpackage
>         config_list = self.get_subpackage(subpackage_name,subpackage_path)
>       File "/home/ink/skipm/src/numpy/numpy/distutils/misc_util.py", line 582, in get_subpackage
>         subpackage_path)
>       File "/home/ink/skipm/src/numpy/numpy/distutils/misc_util.py", line 539, in _get_configuration_from_setup_py
>         config = setup_module.configuration(*args)
>       File "/home/ink/skipm/src/numpy/numpy/setup.py", line 10, in configuration
>         config.add_subpackage('core')
>       File "/home/ink/skipm/src/numpy/numpy/distutils/misc_util.py", line 592, in add_subpackage
>         config_list = self.get_subpackage(subpackage_name,subpackage_path)
>       File "/home/ink/skipm/src/numpy/numpy/distutils/misc_util.py", line 582, in get_subpackage
>         subpackage_path)
>       File "/home/ink/skipm/src/numpy/numpy/distutils/misc_util.py", line 539, in _get_configuration_from_setup_py
>         config = setup_module.configuration(*args)
>       File "numpy/core/setup.py", line 215, in configuration
>         config.add_data_dir('tests')
>       File "/home/ink/skipm/src/numpy/numpy/distutils/misc_util.py", line 636, in add_data_dir
>         self.add_data_files((ds,filenames))
>       File "/home/ink/skipm/src/numpy/numpy/distutils/misc_util.py", line 702, in add_data_files
>         dist.data_files.extend(data_dict.items())
>     AttributeError: 'NoneType' object has no attribute 'extend'
> 
> And finally, a traceback.  What's up with that?

Essentially, the same issue here. Since an earlier step failed, dist.data_files
is still None.

-- 
Robert Kern
robert.kern at gmail.com

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter