[Numpy-discussion] performance matrix multiplication vs. matlab

Sun Jul 19 23:35:49 EDT 2009

Jonathan,

What does "ldd /home/jtaylor/lib/python2.5/site-packages/numpy/linalg/lapack_lite.so"
say ?

You need to make sure that it's using the libraries in /usr/local/lib.
You can remove the ones in /usr/lib or "export
LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH".

Hope it helps.

Best,

N

On Fri, Jul 17, 2009 at 3:57 PM, Jonathan
Taylor<jonathan.taylor at utoronto.ca> wrote:
> Following these instructions I have the following problem when I
> import numpy.  Does anyone know why this might be?
>
> Thanks,
> Jonathan.
>
>>>> import numpy
> Traceback (most recent call last):
>  File "<stdin>", line 1, in <module>
>  File "/home/jtaylor/lib/python2.5/site-packages/numpy/__init__.py",
> line 130, in <module>
>    import add_newdocs
>  File "/home/jtaylor/lib/python2.5/site-packages/numpy/add_newdocs.py",
> line 9, in <module>
>    from lib import add_newdoc
>  File "/home/jtaylor/lib/python2.5/site-packages/numpy/lib/__init__.py",
> line 13, in <module>
>    from polynomial import *
>  File "/home/jtaylor/lib/python2.5/site-packages/numpy/lib/polynomial.py",
> line 18, in <module>
>    from numpy.linalg import eigvals, lstsq
>  File "/home/jtaylor/lib/python2.5/site-packages/numpy/linalg/__init__.py",
> line 47, in <module>
>    from linalg import *
>  File "/home/jtaylor/lib/python2.5/site-packages/numpy/linalg/linalg.py",
> line 22, in <module>
>    from numpy.linalg import lapack_lite
> ImportError: /usr/local/lib/libptcblas.so: undefined symbol: ATL_cpttrsm
>
>
> On Sat, Jun 6, 2009 at 12:59 PM, Chris Colbert<sccolbert at gmail.com> wrote:
>> since there is demand, and someone already emailed me, I'll put what I
>> did in this post. It pretty much follows whats on the scipy website,
>> with a couple other things I gleaned from reading the ATLAS install
>> guide:
>>
>> and here it goes, this is valid for Ubuntu 9.04 64-bit  (# starts a
>> comment when working in the terminal)
>>
>>
>> download lapack 3.2.1 http://www.netlib.org/lapack/lapack.tgz
>> download atlas 3.8.3
>> http://sourceforge.net/project/downloading.php?group_id=23725&filename=atlas3.8.3.tar.bz2&a=65663372
>>
>> create folder  /home/your-user-name/build/atlas   #this is where we build
>> create folder /home/your-user-name/build/lapack #atlas and lapack
>>
>> extract the folder lapack-3.2.1 to /home/your-user-name/build/lapack
>> extract the contents of atlas to /home/your-user-name/build/atlas
>>
>>
>>
>> now in the terminal:
>>
>> # remove g77 and get stuff we need
>> sudo apt-get remove g77
>> sudo apt-get install gfortran
>> sudo apt-get install build-essential
>> sudo apt-get install python-dev
>> sudo apt-get install python-setuptools
>> sudo easy_install nose
>>
>>
>> # build lapack
>> cd /home/your-user-name/build/lapack/lapack-3.2.1
>> cp INSTALL/make.inc.gfortran make.inc
>>
>> gedit make.inc
>> #################
>> #in the make.inc file make sure the line   OPTS = -O2 -fPIC -m64
>> #and    NOOPTS = -O0 -fPIC -m64
>> #the -m64 flags build 64-bit code, if you want 32-bit, simply leave
>> #the -m64 flags out
>> #################
>>
>> cd SRC
>>
>> #this should build lapack without error
>> make
>>
>>
>>
>> # build atlas
>>
>> cd /home/your-user-name/build/atlas
>>
>> #this is simply where we will build the atlas
>> #libs, you can name it what you want
>> mkdir Linux_X64SSE2
>>
>> cd Linux_X64SSE2
>>
>> #need to turn off cpu-throttling
>> sudo cpufreq-selector -g performance
>>
>> #if you don't want 64bit code remove the -b 64 flag. replace the
>> #number 2400 with your CPU frequency in MHZ
>> #i.e. my cpu is 2.53 GHZ so i put 2530
>> ../configure -b 64 -D c -DPentiumCPS=2400 -Fa  -alg -fPIC
>> --with-netlib-lapack=/home/your-user-name/build/lapack/lapack-3.2.1/Lapack_LINUX.a
>>
>> #the configure step takes a bit, and should end without errors
>>
>>  #this takes a long time, go get some coffee, it should end without error
>> make build
>>
>> #this will verify the build, also long running
>> make check
>>
>> #this will test the performance of your build and give you feedback on
>> #it. your numbers should be close to the test numbers at the end
>> make time
>>
>> cd lib
>>
>> #builds single threaded .so's
>> make shared
>>
>> #builds multithreaded .so's
>> make ptshared
>>
>> #copies all of the atlas libs (and the lapack lib built with atlas)
>> #to our lib dir
>> sudo  cp  *.so  /usr/local/lib/
>>
>>
>>
>> #now we need to get and build numpy
>>
>> download numpy 1.3.0
>> http://sourceforge.net/project/downloading.php?group_id=1369&filename=numpy-1.3.0.tar.gz&a=93506515
>>
>> extract the folder numpy-1.3.0 to /home/your-user-name/build
>>
>> #in the terminal
>>
>> cd /home/your-user-name/build/numpy-1.3.0
>> cp site.cfg.example site.cfg
>>
>> gedit site.cfg
>> ###############################################
>> # in site.cfg uncomment the following lines and make them look like these
>> [DEFAULT]
>> library_dirs = /usr/local/lib
>> include_dirs = /usr/local/include
>>
>> [blas_opt]
>> libraries = ptf77blas, ptcblas, atlas
>>
>> [lapack_opt]
>> libraries = lapack, ptf77blas, ptcblas, atlas
>> ###################################################
>> #if you want single threaded libs, uncomment those lines instead
>>
>>
>> #build numpy- should end without error
>> python setup.py build
>>
>> #install numpy
>> python setup.py install
>>
>> cd /home
>>
>> sudo ldconfig
>>
>> python
>>>>import numpy
>>>>numpy.test()   #this should run with no errors (skipped tests and known-fails are ok)
>>>>a = numpy.random.randn(6000, 6000)
>>>>numpy.dot(a, a)     # look at your cpu monitor and verify all cpu cores are at 100% if you built with threads
>>
>>
>> Celebrate with a beer!
>>
>>
>> Cheers!
>>
>> Chris
>>
>>
>>
>>
>>
>> On Sat, Jun 6, 2009 at 10:42 AM, Keith Goodman<kwgoodman at gmail.com> wrote:
>>> On Fri, Jun 5, 2009 at 2:37 PM, Chris Colbert <sccolbert at gmail.com> wrote:
>>>> I'll caution anyone from using Atlas from the repos in Ubuntu 9.04  as the
>>>> package is broken:
>>>>
>>>> https://bugs.launchpad.net/ubuntu/+source/atlas/+bug/363510
>>>>
>>>>
>>>> just build Atlas yourself, you get better performance AND threading.
>>>> Building it is not the nightmare it sounds like. I think i've done it a
>>>> total of four times now, both 32-bit and 64-bit builds.
>>>>
>>>> If you need help with it,  just email me off list.
>>>
>>> That's a nice offer. I tried building ATLAS on Debian a year or two
>>> ago and got stuck.
>>>
>>> Clear out your inbox!
>>> _______________________________________________
>>> Numpy-discussion mailing list
>>> Numpy-discussion at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> Numpy-discussion mailing list
>> Numpy-discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

-- 
Nicolas Pinto
Ph.D. Candidate, Brain & Computer Sciences
Massachusetts Institute of Technology, USA
http://web.mit.edu/pinto