Mailman 3 June 2008 - NumPy-Discussion

ndarray methods vs numpy module functions
by Bob Dowling 24 Jun '08

24 Jun '08

[ I'm new here and this has the feel of an FAQ but I couldn't find anything at http://www.scipy.org/FAQ . If I should have looked somewhere else a URL would be gratefully received. ] What's the reasoning behind functions like sum() and cumsum() being provided both as module functions (numpy.sum(data, axis=1)) and as object methods (data.sum(axis=1)) but other functions - and I stumbled over diff() - only being provided as module functions? >>> print numpy.__version__ 1.1.0 >>> data = numpy.array([[1,2,3],[4,5,6]]) >>> numpy.sum(data,axis=1) array([ 6, 15]) >>> data.sum(axis=1) array([ 6, 15]) >>> numpy.diff(data,axis=1) array([[1, 1], [1, 1]]) >>> data.diff(axis=1) Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'numpy.ndarray' object has no attribute 'diff'

7 12

embedded arrays
by Thomas Hrabe 24 Jun '08

24 Jun '08

Hi all, while writing a extension module in C++ for python & numpy, I find a strange error. I can send and retrieve numpy arrays to and from my module. But python stops if I do the following: a = numpy.array([[1.1,2,3],[4,5,6]]) PM.put(a,'a') //send a to the module b = PM.get('a') //get a identical copy from the module print b array([[ 1.1, 2. , 3. ], [ 4. , 5. , 6. ]]) b.__class__ Out[36]: <type 'numpy.ndarray'> perfect, until a == b the interpreter does not continue from here... I can add values to to b, everything, but a == b simply crashes ...? Does anybody have a clue for this problem? Have a nice weekend, Thomas

3 5

Keywords in wrapped functions
by Pierre GM 24 Jun '08

24 Jun '08

All, Sorry to bumpt the post, accept my apologies for my rudeness, but I'm curious... So, let me rephrase my question: Many numpy functions (min, max, sum...) based on ndarray methods have a construction of the style #----------------------------------------------- def amin(a, axis=None, out=None): try: amin = a.min except AttributeError: return _wrapit(a, 'min', axis, out) return amin(axis, out) #----------------------------------------------- I'm wondering why the initial keywords arguments are transformed into positional arguments in the return. In the previous example, I would expect a line like return amin(axis=axis, out=out) [Actually, I would even prefer a return amin(axis=axis, out=out, **other_optional_parameters), which would permit to add some extra parameters such as a "fill_value", when dealing with masked arrays ?] Is this for efficiency purposes ?

3 3

_md5 module?
by Charles Doutriaux 24 Jun '08

24 Jun '08

Hello on redhat enterprise 5 i get this with numpy 1.1 and python 2.5 Any idea if we can get around tihs? short of rebuilding python with md5 support i guess. Shouldn't numpy catch that before building? C. "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/__init__.py", line 42, in <module> import add_newdocs File "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/add_newdocs.py", line 5, in <module> from lib import add_newdoc File "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/__init__.py", line 18, in <module> from io import * File "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/io.py", line 16, in <module> from _datasource import DataSource File "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/site-packages/numpy/lib/_datasource.py", line 42, in <module> from urllib2 import urlopen, URLError File "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/urllib2.py", line 91, in <module> import hashlib File "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/hashlib.py", line 133, in <module> md5 = __get_builtin_constructor('md5') File "/usr/local/cdat/release/5.0e/5.0.0.alpha7/lib/python2.5/hashlib.py", line 60, in __get_builtin_constructor import _md5 ImportError: No module named _md5

2 2

Sparse Matrices in Numpy -- (with eigenvalue algorithms if possible)
by Dan Yamins 24 Jun '08

24 Jun '08

I'm wondering if there is a currently-available Sparse-Matrix package for numpy? If so, how do I get it? And, if there is a good sparse-matrix package, does it include an eigenvalue-computation algorithm? How would a sparse-matrix package interact with something like numpy.linalg.eig, or for that matter any of the other numpy modules? Thanks, Dan

5 8

Py_NotImplementedType leak
by Charles R Harris 23 Jun '08

23 Jun '08

Leaks of Py_NotImplementedType to the user are regarded as errors by the Python developers and google turns up several patches fixing such issues. In numpy we have this problem because we issue the same call for, say, right_shift as we do for >>. This leads to things like: In [1]: int64(1) >> float32(1) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /home/charris/<ipython console> in <module>() TypeError: unsupported operand type(s) for >>: 'int' and 'numpy.float32' In [2]: right_shift(int64(1),float32(1)) Out[2]: NotImplemented There are several things going on here. First, all the numbers are promoted to 0-D object arrays, so numpy is using the logic for numpy.object scalars. Second, the Python interpreter knows how to deal with the >> operator for Python ints, which has the left and right slots, but can't deal with the explicit call to right_shift because it knows nothing about it. Since proper behavior in this case depends on knowledge of how right_shift is called I think the thing to do is raise a TypeError{NotImplemented} exception in the explicit right_shift call and move the NotImplementedType logic up to the __rrshift__ slot for the numpy array types. This also applies to the other standard numeric methods. Thoughts? Chuck

1 1

set_printoptions - floating point format option?
by Robert Cimrman 23 Jun '08

23 Jun '08

Hi, I need to display some numpy arrays in mantissa+exponent format (e.g. '%.2e' using C syntax). In numpy.set_printoptions(), there is currently only 'precision' option, which does not allow this. What about having an option related to 'precision', named possibly 'float_format', with the following function: :Parameters: format : string Format string for floating point output, has precedence over 'precision' (default '%.8f'). r.

1 0

Travis: this looks like a bug in ufuncobject.c
by Charles R Harris 23 Jun '08

23 Jun '08

select_types(PyUFuncObject *self, int *arg_types, PyUFuncGenericFunction *function, void **data, PyArray_SCALARKIND *scalars, PyObject *typetup) { int i, j; char start_type; int userdef=-1; int userdef_ind=-1; if (self->userloops) { for(i=0; i<self->nin; i++) { if (PyTypeNum_ISUSERDEF(arg_types[i])) { userdef = arg_types[i]; userdef_ind = i; break; } } } if (typetup != NULL) return extract_specified_loop(self, arg_types, function, data, typetup, userdef); if (userdef > 0) { ^^^^^^^^^^^^ PyObject *key, *obj; int ret=-1; obj = NULL; /* Look through all the registered loops for all the user-defined types to find a match. */ while (ret == -1) { if (userdef_ind >= self->nin) break; userdef = arg_types[userdef_ind++]; if (!(PyTypeNum_ISUSERDEF(userdef))) continue; key = PyInt_FromLong((long) userdef); if (key == NULL) return -1; obj = PyDict_GetItem(self->userloops, key); Py_DECREF(key); if (obj == NULL) continue; /* extract the correct function data and argtypes for this user-defined type. */ ret = _find_matching_userloop(obj, arg_types, scalars, function, data, self->nargs, self->nin); } if (ret == 0) return ret; PyErr_SetString(PyExc_TypeError, _types_msg); return ret; } ... It looks like the test should be userdef >= 0. Chuck

2 1

Surprising performance tweak in Cython
by Gael Varoquaux 23 Jun '08

23 Jun '08

I tried tweak my Cython code for performance by manually inlining a small function, and ended up with a less performant code. I must confess I don't really understand what is going on here. If somebody has an explaination, I'd be delighted. The code follows. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ from numpy import zeros # Make sure numpy is initialized. include "c_numpy.pxd" ############################################################################## cdef int inner_loop(float c_x, float c_y): cdef float x, y, x_buffer x = 0; y = 0 cdef int i for i in range(50): x_buffer = x*x - y*y + c_x y = 2*x*y + c_y x = x_buffer if (x*x + x*y > 100): return 50 - i def do_Mandelbrot_cython(): cdef ndarray threshold_time threshold_time = zeros((500, 500)) cdef double *tp cdef float c_x, c_y cdef int i, j c_x = -1.5 tp = <double*>threshold_time.data for i in range(500): c_y = -1 for j in range(500): tp += 1 c_y += 0.004 tp[0] = inner_loop(c_x, c_y) c_x += 0.004 return threshold_time def do_Mandelbrot_cython2(): cdef ndarray threshold_time threshold_time = zeros((500, 500)) cdef double *tp tp = <double*>threshold_time.data cdef float x, y, xbuffer, c_x, c_y cdef int i, j, n c_x = -1.5 for i in range(500): c_y = -1 for j in range(500): tp += 1 c_y += 0.004 x = 0; y = 0 for n in range(50): x_buffer = x*x - y*y + c_x y = 2*x*y + c_y x = x_buffer if (x*x + y*y > 100): tp[0] = 50 -n break c_x += 0.004 return threshold_time ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ And the timing I get are: In [2]: %timeit C.do_Mandelbrot_cython2() 10 loops, best of 3: 342 ms per loop In [3]: %timeit C.do_Mandelbrot_cython() 10 loops, best of 3: 126 ms per loop Cheers, Gaël

3 6

On my Cython/NumPy project
by Dag Sverre Seljebotn 22 Jun '08

22 Jun '08

Since there's been a lot of Cython discussion lately I thought I'd speak up and start a thread specifically for my project. We've just recently set a clear direction for my Google Summer of Code project for integrating NumPy in a better way with Cython. What we ended up with is that I will over the summer add Cython support for the Python 3 buffer interface: http://wiki.cython.org/enhancements/buffer (We aim to support NumPy back to Python 2.3, perhaps as a special case, but the buffer interface will guide in what way we support it.) General feedback is welcome; in particular, I need more opinions about what syntax people would like. We seem unable to find something that we really like; this is the current best candidate (cdef is the way you declare types on variables in Cython): cdef int i = 4, j = 6 cdef np.ndarray[np.float64, 2] arr = np.zeros((10, 10), dtype=np.float64) arr[i, j] = 1 ... The code above the under the hood acquires a buffer and uses it for efficient access. arr[5], arr[5,6,7], arr[2:4,...] and in general anything but two simple indices will be passed to Python, while arr[i, j] will be passed to the buffer. This is a natural first step and I'll focus on this exclusively in the beginning. (That said, once this is in place, there are of course lots of opportunities of doing an even better job in supporting the numerical community with Cython: ndenumerate, more efficient buffer acquisition and array creation, more efficient calling of NumPy/SciPy functions without going through the Python "calling convention", etc..) Dag Sverre

9 19