Mailman 3 November 2010 - NumPy-Discussion

numpy + amdlibm?
by Neal Becker 18 Nov '10

18 Nov '10

Anyone tried building numpy with amdlibm?

1 0

Re: [Numpy-discussion] RuntimeWarning: Item size computed from the PEP 3118
by Laurent Gautier 18 Nov '10

18 Nov '10

Thanks for the reply. On 11/17/10 7:00 PM, Pauli Virtanen<pav(a)iki.fi> wrote: > > Wed, 17 Nov 2010 08:40:50 +0100, Laurent Gautier wrote: > [clip] >> > __main__:1: RuntimeWarning: Item size computed from the PEP 3118 buffer >> > format string does not match the actual item size. > [clip] >> > I tried tracing a bit further but I am a little confused by what is the >> > intent (a lot of nested calls). > Numpy is warning you that the view that you passed to it is inconsistent, > and it indicates that it refuses to consider it as a PEP 3118 buffer. If > you use it on Python 2.x, it probably falls back to using the legacy > buffer interface which does not care about the format string. That would be the __array_struct__ attribute, I suppose. >> > my buffer format is 'f', and the view has itemsize set as: >> > view->itemsize = sizeof(double); >> > >> > Any pointer regarding what might be going on ? > Seehttp://docs.python.org/library/struct.html#format-strings > > The format 'f' means 32-bit float, so the view itemsize should be > sizeof(float) and not sizeof(double). If your view really contains floats > in sizeof(double) bytes, the buffer format string probably should > indicate the padding. It seems that I have been mislead by some documentation (which I cannot find again). I had something that 'f' indicated a "Python float", which I understand to be represented by a double at C-level. Laurent > -- Pauli Virtanen

1 0

half-float review/pull request
by Mark Wiebe 18 Nov '10

18 Nov '10

I've created a ticket and a pull request for the half-float type I implemented. Since my previous email, I've moved the half implementation into npymath so modules using numpy can do half-float code as well. There are a few key things that in my mind should get some discussion before saying this is final: * The character code for half. I chose 'j' because 'h' was already taken by the 16-bit integer type, but maybe there's a reason to choose a different letter. * The API in npymath (or libndarray once refactoring is merged). I've tried to make it clean and follow numpy or ieee-754 conventions, but maybe it could be improved a bit. Here's the pull request: https://github.com/numpy/numpy/pull/16 Thanks, Mark

1 0

Regrading Numpy Documentation ...
by srinivas zinka 18 Nov '10

18 Nov '10

Hi, I downloaded the "Numpy reference guide" in HTML format from the following link: http://docs.scipy.org/doc/ My intension is to use this documentation in "offline mode". But, in offline mode, I am unable to search the document using "quick search" option. (However, I can search the same document on the documentation website). I would like to know, if there is any other way to search the entire HTML reference guide in offline mode. Thank you, zinka

4 7

unicode string for specifying dtype
by Antony Lee 18 Nov '10

18 Nov '10

I just ran into the following: >>> np.dtype(u"f4") Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: data type not understood Is that the expected behaviour? Thanks in advance, Antony Lee

4 4

[ANN] Blosc 1.0.3 released
by Francesc Alted 17 Nov '10

17 Nov '10

==================================================== Announcing python-blosc 1.0.3 A Python wrapper for the Blosc compression library ==================================================== What is it? =========== Blosc (http://blosc.pytables.org) is a high performance compressor optimized for binary data. It has been designed to transmit data to the processor cache faster than the traditional, non-compressed, direct memory fetch approach via a memcpy() OS call. Blosc works well for compressing numerical arrays that contains data with relatively low entropy, like sparse data, time series, grids with regular-spaced values, etc. python-blosc is a Python package that wraps it. What is new? ============ Blosc has been updated to 1.1.3, allowing much improved compression ratio under some circumstances. Also, the number of cores on Windows platform is detected correctly now (thanks to Han Genuit). Last, but not least, Windows binaries for Python 2.6 and 2.7 are provided (both in 32-bit and 64-bit flavors). For more info, you can see the release notes in: https://github.com/FrancescAlted/python-blosc/wiki/Release-notes Basic Usage =========== # Create a binary string made of int (32-bit) elements >>> import array >>> a = array.array('i', range(10*1000*1000)) >>> bytes_array = a.tostring() # Compress it >>> import blosc >>> bpacked = blosc.compress(bytes_array, typesize=a.itemsize) >>> len(bytes_array) / len(bpacked) 110 # 110x compression ratio. Not bad! # Compression speed? >>> from timeit import timeit >>> timeit("blosc.compress(bytes_array, a.itemsize)", "import blosc, array; \ a = array.array('i', range(10*1000*1000)); \ bytes_array = a.tostring()", \ number=10) 0.040534019470214844 >>> len(bytes_array)*10 / 0.0405 / (1024*1024*1024) 9.1982476505232444 # wow, compressing at ~ 9 GB/s. That's fast! # This is actually much faster than a `memcpy` system call >>> timeit("ctypes.memmove(b.buffer_info()[0], a.buffer_info()[0], \ len(a)*a.itemsize)", "import array, ctypes; \ a = array.array('i', range(10*1000*1000)); \ b = a[::-1]", number=10) 0.10316681861877441 >>> len(bytes_array)*10 / 0.1031 / (1024*1024*1024) 3.6132786600018565 # ~ 3 GB/s is memcpy speed # Decompress it >>> bytes_array2 = blosc.decompress(bpacked) # Check whether our data have had a good trip >>> bytes_array == bytes_array2 True # yup, it seems so # Decompression speed? >>> timeit("s2 = blosc.decompress(bpacked)", "import blosc, array; \ a = array.array('i', range(10*1000*1000)); \ bytes_array = a.tostring(); \ bpacked = blosc.compress(bytes_array, a.itemsize)", \ number=10) 0.083872079849243164 > len(bytes_array)*10 / 0.0838 / (1024*1024*1024) 4.4454538167803275 # decompressing at ~ 4.4 GB/s is pretty good too! [Using a machine with 8 physical cores with hyper-threading] The above examples use maximum compression level 9 (default), and although lower compression levels produce smaller compression ratios, they are also faster (reaching speeds exceeding 11 GB/s). More examples showing other features (and using NumPy arrays) are available on the python-blosc wiki page: http://github.com/FrancescAlted/python-blosc/wiki Documentation ============= Please refer to docstrings. Start by the main package: >>> import blosc >>> help(blosc) and ask for more docstrings in the referenced functions. Download sources ================ Go to: http://github.com/FrancescAlted/python-blosc and download the most recent release from here. Blosc is distributed using the MIT license, see LICENSES/BLOSC.txt for details. Mailing list ============ There is an official mailing list for Blosc at: blosc(a)googlegroups.com http://groups.google.es/group/blosc ---- **Enjoy data!** -- Francesc Alted

1 0

Python scalar float indices into array work - but not for array indices - why ?
by Sebastian Haase 17 Nov '10

17 Nov '10

Hi, >>> import numpy as np >>> a = np.arange(4) >>> a[1.8] 1 >>> a[ np.array(1.8) ] Traceback (most recent call last): File "<input>", line 1, in <module> IndexError: arrays used as indices must be of integer (or boolean) type >>> Why does numpy not accept float arrays as indices ? I was very happy and quite surprised once I found out that it worked at all for Python float scalars, but would it not just be consequent to also allow float ndarrays then ? Regards, Sebastian Haase

3 7

RuntimeWarning: Item size computed from the PEP 3118
by Laurent Gautier 17 Nov '10

17 Nov '10

Hi, I am developping a package using the buffer interface, and with Python 2.7 - Numpy 1.5, the following annoying warning has been reported. __main__:1: RuntimeWarning: Item size computed from the PEP 3118 buffer format string does not match the actual item size. Beside warning it appears that all is fine. The source of the warning can be traced down the C level with the following boolean expression being true: descr->elsize != view->itemsize I tried tracing a bit further but I am a little confused by what is the intent (a lot of nested calls). my buffer format is 'f', and the view has itemsize set as: view->itemsize = sizeof(double); Any pointer regarding what might be going on ? Thanks, Laurent

2 1

"numpy.linalg.linalg.LinAlgError: Singular matrix" using "numpy.linalg.solve"
by SoTaNeZ 16 Nov '10

16 Nov '10

Hi all. I got this exception while executin numpy.linalg.solve(a,b) being: a = array([[ 1.00000000e+000, -4.19430400e+006, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000], [ 1.00000000e+000, 0.00000000e+000, -2.47845883e-119, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000], [ 1.00000000e+000, 0.00000000e+000, 0.00000000e+000, -2.68435456e+008, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000], [ 1.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, -3.38813179e-021, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000], [ 1.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, -4.09600000e+003, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000], [ 1.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, -2.56000000e+002, 0.00000000e+000, 0.00000000e+000], [ 1.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, -7.20575940e+016, 0.00000000e+000], [ 1.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, 0.00000000e+000, -1.56250000e-002], [ 1.00000000e+000, 1.00000000e+000, 1.00000000e+000, 1.00000000e+000, 1.00000000e+000, 1.00000000e+000, 1.00000000e+000, 1.00000000e+000, 1.00000000e+000]]) b = array([0, 0, 0, 0, 0, 0, 0, 0, 1]) I guess some numbers in a are too big or too small, aren't they? Thanks, David.

2 1

Workaround for Ticket #1504 (MKL linking)
by Felix 16 Nov '10

16 Nov '10

is there any workaround or fix for the problem described in Ticket 1504? http://projects.scipy.org/numpy/ticket/1504 Using static linking sounds like it could be the easiest solution. Can numpy.distutils be used to do that? Thank you for any tips Felix

3 5

NumPy-Discussion November 2010