Mailman 3 July 2014 - NumPy-Discussion

Fast way to convert (nested) list to numpy object array?
by Marc Hulsman July 5, 2014

July 5, 2014

Hello, In my application I use nested, someitmes variable length lists, e.g. [[1,2], [1,2,3], ...]. These can also become double nested, etc. up to arbitrary complexity. I like to use numpy indicing on the outer list, i.e. I want to create: array([[1, 2], [1, 2, 3]], dtype=object) However, because numpy likes to 'walk' through the nested lists, this becomes rather slow when the nested lists are large, e.g. k = [range(i) for i in range(10000)] %timeit numpy.array(k) 1 loops, best of 3: 2.11 s per loop Compared to shorter lists, e.g: k2 = [range(numpy.random.randint(0,10)) for i in range(10000)] %timeit numpy.array(k2) 100 loops, best of 3: 2.7 ms per loop As I know beforehand that numpy does not have to descend into these objects, I would just like to create a 1-dimensional array. I thought about using fromiter, but his fails with: ValueError: cannot create object arrays from iterator A second approach I tried is to create an empty array, and then fill it: x = numpy.empty(len(k), dtype=object) %timeit x[:] = k 1000 loops, best of 3: 220 µs per loop This works already much, much better, but the loop still takes time to 'descend' into the objects if they have a fixed size, e.g.: k3 = [[range(10) for i in range(100)] for i in range(10000)] %timeit x[:] = k3 10 loops, best of 3: 45.6 ms per loop A python loop is in these cases even faster %timeit for pos, e in enumerate(k3): x[pos] = e 1000 loops, best of 3: 1.02 ms per loop This piece of code is quite time-critical in my application, and I observe slow downs due to this behaviour. My question therefore is if there is a fast way to just convert a list simply into a 1-dimensional object array, without each object being descended into? More in general, if i create an array with numpy.array(k), would it be possible to indicate that it should search only 1,2,... nested levels deep into k? Thanks for any advice, Marc

5 8

Numpy and debug symbols
by Pablo Pérez García July 3, 2014

July 3, 2014

Hello, I'm a newcomer and I have a question I did not manage to solve yet, I posted it into these two stack-overflow entries: http://stackoverflow.com/questions/24529811/compiling-numpy-for-windows-pyt… http://stackoverflow.com/questions/24548485/using-numpy-on-an-embedded-pyth… Thank you very much in advance! -- Pablo Pérez García

3 4

Accessing irregular sized array data from C
by Mads Ipsen July 2, 2014

July 2, 2014

Hi, If you setup an M x N array like this a = 1.0*numpy.arange(24).reshape(8,3) you can access the data from a C function like this void foo(PyObject * numpy_data) { // Get dimension and data pointer int const m = static_cast<int>(PyArray_DIMS(numpy_data)[0]); int const n = static_cast<int>(PyArray_DIMS(numpy_data)[1]); double * const data = (double *) PyArray_DATA(numpy_data); // Access data ... } Now, suppose I have an irregular shaped numpy array like this a1 = numpy.array([ 1.0, 2.0, 3.0]) a2 = numpy.array([-2.0, 4.0]) a3 = numpy.array([5.0]) b = numpy.array([a1,a2,a3]) How can open up the doors to the array data of b on the C-side? Best regards, Mads -- +---------------------------------------------------------+ | Mads Ipsen | +----------------------+----------------------------------+ | Gåsebæksvej 7, 4. tv | phone: +45-29716388 | | DK-2500 Valby | email: mads.ipsen(a)gmail.com | | Denmark | map : www.tinyurl.com/ns52fpa | +----------------------+----------------------------------+

5 8

numpy.histogram not giving expected results
by Moroney, Catherine M (398D) July 2, 2014

July 2, 2014

Hello, I'm trying to calculate a 1-d histogram of a distribution that contains mostly zeros, and I'm having problems with examples where the values to be histogrammed fall exactly on the bin boundaries: For example, this gives me the expected results (entering the exact bin values): >>> data array([ 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0.05, -0.05]) >>> bins_list = numpy.array([-0.1, -0.05, 0.0, 0.05, 0.1]) >>> (counts, edges) = numpy.histogram(data, bins=bins_list) >>> counts array([ 0, 1, 10, 1]) >>> edges array([-0.1 , -0.05, 0. , 0.05, 0.1 ]) but this does not (generating the bin values via bumpy.arange): >>> bins_arange = numpy.arange(-0.1, 0.101, 0.05) >>> data array([ 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0.05, -0.05]) >>> bins_arange array([-0.1 , -0.05, 0. , 0.05, 0.1 ]) >>> (counts, edges) = numpy.histogram(data, bins=bins_arange) >>> counts array([ 0, 1, 11, 0]) I'm assuming this is due to slight rounding in the calculation of bins_arange, as compared to the manually entered values in bins_list. What is the recommended way of getting the first set of results, without having to manually enter all the values in the "bins" argument? The following also gives me unexpected results: >>> data array([ 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. , 0.05, -0.05]) counts, edges) = numpy.histogram(data, range=(-0.1, 0.1), bins=4) >>> counts array([ 0, 1, 11, 0]) Thank you for any advice, Catherine

4 7