Mailman 3 March 2013 - NumPy-Discussion

Generalized inner?
by Jaime Fernández del Río 25 Mar '13

25 Mar '13

The other day I found myself finding trailing edges in binary images doing something like this: arr = np.random.randint(2, size=1000).astype(np.int8) pattern = np.array([1, 1, 1, 1, 0, 0]) arr_match = 2*arr - 1 pat_match = 2*pattern - 1 from numpy.lib.stride_tricks import as_strided arr_win = as_strided(arr_match, shape=arr.shape[:-1] + (arr.shape[-1]-len(pattern)+1, len(pattern)), strides=arr.strides+arr.strides[-1:]) matches = np.einsum('...i, i', arr_win, pat_match) == len(pattern) While this works fine, this led me to thinking that all this functions (inner, dot, einsum, tensordot...) could be generalized to any other ufuncs apart from a pointwise np.multiply followed by an np.add reduction. It would be great if there was a np.gen_inner that allowed something like: np.gen_inner(arr_win, pattern, pointwise=np.equal, reduce=np.logical_and) I would like to think that such a generalization would be useful in other settings (although I can't think of any right now), and that it could find it's place in numpy, rather than in scipy.ndimage or the like. Does this make any sense? Is there any already existing way of doing this that I'm overlooking? Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

2 1

Numpy 1.7.1
by Charles R Harris 24 Mar '13

24 Mar '13

The Numpy 1.7.1 release process seems to have stalled. What do we need to finish up to get it going again? I think it would be nice to shoot for a release maybe the weekend after next. Chuck

4 4

C-API: Subclassing array for numpy 1.7 (with #define NPY_NO_DEPRECATED_API 0x00000007)
by Marc Gronle 24 Mar '13

24 Mar '13

Hello together, we embedded Python 3 in a C++ environment. In this application I created a new class, that is a subclass from numpy array. Until now (numpy 1.6 or numpy 1.7 without the deprecation define (NPY_NO_DEPRECATED_API) the typedef for my class describing the object was something like typedef struct { PyArrayObject numpyArray; int myMember1; int myMember2; ... } PySubclassObject This always worked for me and a call to PyArray_NDIM(obj) returned the number of dimensions, where obj is of type PySubclassObject*. If I am now using numpy 1.7 this also works if the line #define NPY_NO_DEPRECATED_API 0x00000007 is not set. However, if I removed all deprecated content, there occurrs the first runtime error when creating an instance of PySubclassObject. This is due to the fact, that PyArrayObject now only is a tiny typedef of the following form typedef struct tagPyArrayObject { PyObject_HEAD } PyArrayObject; Previously this has been something like typedef struct tagPyArrayObject { PyObject_HEAD char *data; int nd; npy_intp *dimensions; ... } PyArrayObject; Usually, when creating only one np.array, there is extra space allocated depending on the size of PyArrayObject_fields. However, in my subclass I don't know how to add that extra space between the members numpyArray and myMember1. Finally, when calling PyArray_NDIM(obj) like above, the obj-pointer is casted in the macro to PyArrayObject_fields*. This yields an access conflict with myMember1, myMember2... and the members of PyArrayObject_fields. I hope this description was clear enough to understand my problem. Has anybody an idea what I need to change such that the subclassing also works for the new numpy structure? Thanks for any answer. Cheers Marc

1 0

Execution time difference between 2.7 and 3.2 using numpy
by Colin J. Williams 24 Mar '13

24 Mar '13

8 19

Problem with numpy.records module
by Pierre Barbier de Reuille 22 Mar '13

22 Mar '13

Hello, I am trying to use titles for record arrays. In the documentation, it is specified that any column can set to "None". However, using 'None' fails on numpy 1.6.2 because in np.core.records, on line 195, the "strip" method is called on the title object. First, I hope I understood the documentation correctly. If so, is it possible to replace the line 195 with: self._titles = [n.strip() if n is not None else None for n in titles[:self._nfields]] so 'None' elements are handled properly? Thanks, -- Barbier de Reuille Pierre

1 0

Videos of PyCon talks.
by Charles R Harris 22 Mar '13

22 Mar '13

Here <http://pyvideo.org/category/33/pycon-us-2013>. Chuck

1 0

numpy array to C API
by Søren 21 Mar '13

21 Mar '13

Greetings I'm extending our existing C/C++ software with Python/Numpy in order to do extra number crunching. It already works like a charm calling python with the C API <Python.h>. But what is the proper way of passing double arrays returned from Python/Numpy routines back to C? I came across PyArray but I can see in the compiler warnings, it is deprecated and I don't wanna start from scratch on legacy facilities. Going forward, what is the intended way of doing this with neat code on both sides and with a minimum of mem copy gymnastics overhead? thanks in advance Søren

5 8

how to efficiently select multiple slices from an array?
by Andreas Hilboll 20 Mar '13

20 Mar '13

Cross-posting a question I asked on SO (http://stackoverflow.com/q/15527666/152439): Given an array d = np.random.randn(100) and an index array i = np.random.random_integers(low=3, high=d.size - 5, size=20) how can I efficiently create a 2d array r with r.shape = (20, 8) such that for all j=0..19, r[j] = d[i[j]-3:i[j]+5] In my case, the arrays are quite large (~200000 instead of 100 and 20), so something quick would be useful. Cheers, Andreas.

2 2

Add ability to disable the autogeneration of the function signature in a ufunc docstring.
by Warren Weckesser 20 Mar '13

20 Mar '13

Hi all, In a recent scipy pull request (https://github.com/scipy/scipy/pull/459), I ran into the problem of ufuncs automatically generating a signature in the docstring using arguments such as 'x' or 'x1, x2'. scipy.special has a lot of ufuncs, and for most of them, there are much more descriptive or conventional argument names than 'x'. For now, we will include a nicer signature in the added docstring, and grudgingly put up with the one generated by the ufunc. In the long term, it would be nice to be able to disable the automatic generation of the signature. I submitted a pull request to numpy to allow that: https://github.com/numpy/numpy/pull/3149 Comments on the pull request would be appreciated. Thanks, Warren

3 5

Dot/inner products with broadcasting?
by Jaakko Luttinen 20 Mar '13

20 Mar '13

Hi! How can I compute dot product (or similar multiply&sum operations) efficiently so that broadcasting is utilized? For multi-dimensional arrays, NumPy's inner and dot functions do not match the leading axes and use broadcasting, but instead the result has first the leading axes of the first input array and then the leading axes of the second input array. For instance, I would like to compute the following inner-product: np.sum(A*B, axis=-1) But numpy.inner gives: A = np.random.randn(2,3,4) B = np.random.randn(3,4) np.inner(A,B).shape # -> (2, 3, 3) instead of (2, 3) Similarly for dot product, I would like to compute for instance: np.sum(A[...,:,:,np.newaxis]*B[...,np.newaxis,:,:], axis=-2) But numpy.dot gives: In [12]: A = np.random.randn(2,3,4); B = np.random.randn(2,4,5) In [13]: np.dot(A,B).shape # -> (2, 3, 2, 5) instead of (2, 3, 5) I could use einsum for these operations, but I'm not sure whether that's as efficient as using some BLAS-supported(?) dot products. I couldn't find any function which could perform this kind of operations. NumPy's functions seem to either flatten the input arrays (vdot, outer) or just use the axes of the input arrays separately (dot, inner, tensordot). Any help? Best regards, Jaakko

2 5