[Numpy-discussion] Help building numpy
Bruce Southey
bsouthey at gmail.com
Fri Oct 8 12:35:57 EDT 2010
On 10/08/2010 10:58 AM, Ian Goodfellow wrote:
> Here's the output on my atlas library:
> file -L /usr/local/atlas/lib/libatlas.so
> /usr/local/atlas/lib/libatlas.so: ELF 64-bit LSB shared object,
> x86-64, version 1 (SYSV), dynamically linked, not stripped
>
> It looks mostly the same as yours except it says "not stripped" while
> yours says "stripped." Do you know what that means?
>
> Here's my timing code (but the difference between installations is
> pretty obvious even without a timer, 8 full seconds on my machine
> versus .2 seconds on a slower machine at work):
>
> import numpy as N
> import time
> rng = N.random.RandomState([0,1,2])
> A = rng.randn(1000,1000)
> t1 = time.time(); x = N.dot(A,A); t2 = time.time()
> print t2-t1
>
>
>
>
> On Fri, Oct 8, 2010 at 11:49 AM, Bruce Southey <bsouthey at gmail.com
> <mailto:bsouthey at gmail.com>> wrote:
>
> On 10/08/2010 10:01 AM, Ian Goodfellow wrote:
>> I'm using 64-bit Ubuntu 10.04. I originally tried building
>> without site.cfg and got the same result. After that I removed
>> the installation and the numpy/build directory and tried again
>> with site.cfg.
>>
>> Here's the otuput of my show_config(). Does this mean it's
>> actually using atlas? I ran 'make time' in the atlas directory
>> and got higher % of clock speed numbers than the reference so I
>> would be surprised if my atlas build is this slow.
>>
>> atlas_threads_info:
>> libraries = ['lapack', 'lapack', 'f77blas', 'cblas', 'atlas']
>> library_dirs = ['/usr/local/atlas/lib']
>> define_macros = [('ATLAS_INFO', '"\\"None\\""')]
>> language = f77
>> include_dirs = ['/usr/include']
>> blas_opt_info:
>> libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
>> library_dirs = ['/usr/local/atlas/lib']
>> define_macros = [('ATLAS_INFO', '"\\"None\\""')]
>> language = c
>> include_dirs = ['/usr/include']
>> atlas_blas_threads_info:
>> libraries = ['lapack', 'f77blas', 'cblas', 'atlas']
>> library_dirs = ['/usr/local/atlas/lib']
>> define_macros = [('ATLAS_INFO', '"\\"None\\""')]
>> language = c
>> include_dirs = ['/usr/include']
>> lapack_opt_info:
>> libraries = ['lapack', 'lapack', 'f77blas', 'cblas', 'atlas']
>> library_dirs = ['/usr/local/atlas/lib']
>> define_macros = [('ATLAS_INFO', '"\\"None\\""')]
>> language = f77
>> include_dirs = ['/usr/include']
>> lapack_mkl_info:
>> NOT AVAILABLE
>> blas_mkl_info:
>> NOT AVAILABLE
>> mkl_info:
>> NOT AVAILABLE
>>
>>
>> On Fri, Oct 8, 2010 at 10:53 AM, Bruce Southey
>> <bsouthey at gmail.com <mailto:bsouthey at gmail.com>> wrote:
>>
>> On 10/08/2010 09:06 AM, Benjamin Root wrote:
>>> On Fri, Oct 8, 2010 at 8:47 AM, Ian Goodfellow
>>> <goodfellow.ian at gmail.com <mailto:goodfellow.ian at gmail.com>>
>>> wrote:
>>>
>>> Can anyone explain how to get numpy to recognize atlas?
>>> I have atlas built and installed, and I put what I
>>> thought should be sufficient, based on numpy's
>>> INSTALL.TXT, to make numpy use atlas in my site.cfg then
>>> ran python setup.py install. The resulting build is
>>> incredibly slow. Multiplying a 1,000 x 1,000 matrix
>>> takes 8 seconds. The default packages take .8 . On a
>>> computer at work with a similar but slower cpu to mine,
>>> it only takes .2, so I know I should be able to get a
>>> lot more speed out of numpy.
>>>
>>> This is my site.cfg:
>>>
>>> [atlas]
>>> atlas_libs = lapack, f77blas, cblas, atlas
>>>
>>> [DEFAULT]
>>> library_dirs = /usr/local/atlas/lib
>>> include_dir = /usr/local/atlas/include
>>>
>>> Thanks in advance,
>>> Ian
>>>
>>>
>>> Ian, it would be very helpful if you could post the output
>>> from the build of numpy. That log can tell us whether it
>>> found the libraries or not when building.
>>>
>>> Ben Root
>>>
>>>
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at scipy.org
>>> <mailto:NumPy-Discussion at scipy.org>
>>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>> I do not use a 'site.cfg' even though I build my own numpy
>> but use the standard Fedora (13) packages for atlas. So try
>> it without defining it - remove the installation and build
>> directories first!
>>
>> Can you please provide the operating system (including 32bit
>> or 64 bit), Python version and numpy version?
>>
>> What is the output of the numpy function 'show_config()'?
>> I added mine below from a 64-bit linux system so you can see
>> that numpy is looking in the '/usr/lib64' directory.
>>
>> Bruce
>>
>> >>> import numpy as np
>> >>> np.show_config()
>> atlas_threads_info:
>> libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
>> library_dirs = ['/usr/lib64/atlas']
>> define_macros = [('ATLAS_INFO', '"\\"3.8.3\\""')]
>> language = f77
>> include_dirs = ['/usr/include']
>> blas_opt_info:
>> libraries = ['ptf77blas', 'ptcblas', 'atlas']
>> library_dirs = ['/usr/lib64/atlas']
>> define_macros = [('ATLAS_INFO', '"\\"3.8.3\\""')]
>> language = c
>> include_dirs = ['/usr/include']
>> atlas_blas_threads_info:
>> libraries = ['ptf77blas', 'ptcblas', 'atlas']
>> library_dirs = ['/usr/lib64/atlas']
>> define_macros = [('ATLAS_INFO', '"\\"3.8.3\\""')]
>> language = c
>> include_dirs = ['/usr/include']
>> lapack_opt_info:
>> libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas']
>> library_dirs = ['/usr/lib64/atlas']
>> define_macros = [('ATLAS_INFO', '"\\"3.8.3\\""')]
>> language = f77
>> include_dirs = ['/usr/include']
>> lapack_mkl_info:
>> NOT AVAILABLE
>> blas_mkl_info:
>> NOT AVAILABLE
>> mkl_info:
>> NOT AVAILABLE
>> >>>
>>
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org <mailto:NumPy-Discussion at scipy.org>
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
>>
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org <mailto:NumPy-Discussion at scipy.org>
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
> It appears to finding something but I do not know enough about
> that aspect.
>
> I presume that you installed your own version of atlas since you
> are using /usr/local. So you need to be careful of the flags used
> especially that it is 64-bit and uses the appropriate SSE
> instructions for your processor. For example, on my system:
>
> $ file -L /usr/lib64/atlas/libatlas.so
> /usr/lib64/atlas/libatlas.so: ELF 64-bit LSB shared object,
> x86-64, version 1 (SYSV), dynamically linked, stripped
>
> Can you please provide the code you are using for timing for the
> 'slow' system?
>
> By the way, there was this thread:
> 'HOWTO build NumPy without external math libs'
> http://mail.scipy.org/pipermail/numpy-discussion/2010-July/051513.html
>
> BLAS=None LAPACK=None ATLAS=None python setup.py build
>
>
> Bruce
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org <mailto:NumPy-Discussion at scipy.org>
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
Don't use time!
Use timeit.
I suspect your build is using numerical libraries that have not been
build correctly for your system. But I don't know anything about that.
What are the flags used to build lapack and atlas?
I told the lazy way of using timeit but may help you identify the slowness.
To capture the output I use two files for stdout and stderr:
$ python setup.py build > b.log 2>e.log
$ python setup.py install # done as root
1) Using numpy built as:
'BLAS=None LAPACK=None ATLAS=None python setup.py build'
$ python -m timeit -s "import numpy as N; rng =
N.random.RandomState([0,1,2]) ; A = rng.randn(1000,1000); x = N.dot(A,A)"
100000000 loops, best of 3: 0.012 usec per loop
2) Using numpy built just without Atlas:
'ATLAS=None python setup.py build'
$ python -m timeit -s "import numpy as N; rng =
N.random.RandomState([0,1,2]) ; A = rng.randn(1000,1000); x = N.dot(A,A)"
100000000 loops, best of 3: 0.012 usec per loop
3) Using Atlas without dot:
$ python -m timeit -s "import numpy as N; rng =
N.random.RandomState([0,1,2]) ; A = rng.randn(1000,1000)"
100000000 loops, best of 3: 0.012 usec per loop
4) Using Atlas with dot:
$ python -m timeit -s "import numpy as N; rng =
N.random.RandomState([0,1,2]) ; A = rng.randn(1000,1000); x = N.dot(A,A)"
100000000 loops, best of 3: 0.0131 usec per loop
Bruce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20101008/33dac12f/attachment.html>
More information about the NumPy-Discussion
mailing list