Mailman 3 February 2009 - NumPy-Discussion

Core math library in numpy
by David Cournapeau Feb. 26, 2009

Feb. 26, 2009

Hi, While trying to fix some issues on windows 64 bits related to the float format fixes, I encountered some problems (lack of some basic math functions, in particular isnan and co for the multiarray module). We already have the portable replacing code for those functions (umath_funcs_c99.inc.src), which is used for the umath module. For now, we can only reuse this implementation by direct inclusion. I think it would be better to have a separate library with those core math routines: - sharing by copying means more code, more compilation time. - it cannot be shared outside numpy core. For some code, we don't have a choice, but for math functions, I don't really see an argument for not sharing: having public, portable math functions sounds useful to me. The only requirement is that we would need to mangle the name, I think, to avoid clash with the C math library on the platforms who have the maths. Would there be any other drawback ? Would people be against this ? cheers, David

7 26

denormal test: what is it for ?
by David Cournapeau Feb. 25, 2009

Feb. 25, 2009

Hi, I would like to continue cleaning the setup.py from numpy/core, and there is one test that I can't make sense of: the denormal thing (testcode_mathlib function). The svn log has no information on it (its presence goes back to the first revision of the file). What is it useful for ? Denormal support ? Whether exp works ? cheers, David

2 1

Problem looping over numpy array in C
by Sumant S.R. Oemrawsingh Feb. 25, 2009

Feb. 25, 2009

Hi guys, I have a problem with looping over numpy arrays in C. I modify the array in-place (and return None), but after modification, the array doesn't seem to play nice any more. Below, I have the C code for a function do_something (a stripped version of my original function), which has as arguments a 1D numpy array (float or complex) and either an int or a sequence of ints. In python, I do the following using only the integer: >>> a = array([0., 1., 2., 3., 2., 1., 0.]) >>> do_something(a,3) >>> savetxt('bla',a) >>> Which works fine. However, when I do the same, but with a list of any length larger than 0: >>> a = array([0., 1., 2., 3., 2., 1., 0.]) >>> do_something(a,[3]) >>> savetxt('bla',a) Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python2.5/site-packages/numpy/core/numeric.py", line 767, in savetxt X = asarray(X) File "/usr/lib/python2.5/site-packages/numpy/core/numeric.py", line 132, in asarray return array(a, dtype, copy=False, order=order) TypeError: an integer is required >>> savetxt('bla',a) >>> For some reason, the first time it doesn't work; the asarray fails but then succeeds on a second try. The resulting file is identical to when I would have looped over the list in python, and called the do_something function with each integer separately. For instance: for i in [3,1,0]: do_something(a, i) works fine as well. So apparently looping in C temporarily leaves the array in a weird state but manages to automatically be restored after one exception. I've checked that type(a), a.dtype, a.shape and a.ndim remain the same before and after calling do_something with a sequence or with an integer as second argument. That doesn't seem to be the problem. The reason that I want do the loop in C is that I need some precalculations, before being able to do the actual loop. But they might not be time-consuming enough to warrant writing it in C, so I could just do the loop in python and not have this problem any more. However, if the loop in C manages to somehow (temporarily) corrupt the array, how can I be sure that the single-shot call doesn't succeed by accident? If anyone can help, suggest something to try or spot my mistake, I'd appreciate it. Thanks, Sumant static PyObject* do_something(PyObject *self, PyObject *args, PyObject *kwargs) { PyObject *input, *s, *item; PyArrayObject *array = NULL; npy_intp i,n; static char *kwlist[] = {"a", "index", NULL}; if (!PyArg_ParseTupleAndKeywords(args, kwargs, "Oi", kwlist, &input, &i)) goto sequence; if ((array = (PyArrayObject *) PyArray_FromAny(input, NULL, 1, 1, NPY_CARRAY | NPY_UPDATEIFCOPY, NULL)) == NULL) goto error; n = PyArray_DIM(array,0); if (PyArray_ISCOMPLEX(array)) do_something_complex((complex*)(PyArray_DATA(array)), n, i); else if (PyArray_ISFLOAT(array)) do_something_double((double*)(PyArray_DATA(array)), n, i); else goto error; Py_DECREF(array); Py_RETURN_NONE; sequence: if (!PyArg_ParseTupleAndKeywords(args, kwargs, "OO", kwlist, &input, &s)) goto error; if (PySequence_Check(s) == 0) goto error; if ((array = (PyArrayObject *) PyArray_FromAny(input, NULL, 1, 1, NPY_CARRAY | NPY_UPDATEIFCOPY, NULL)) == NULL) goto error; n = PyArray_DIM(array,0); if (PyArray_ISCOMPLEX(array)) { complex* data = (complex*)PyArray_DATA(array); for (i = 0; i < PySequence_Size(s); i++) { item = PySequence_GetItem(s, i); do_something_complex(data, n, PyInt_AsSsize_t(item)); Py_DECREF(item); } } else if (PyArray_ISFLOAT(array)) { double* data = (double*)PyArray_DATA(array); for (i = 0; i < PySequence_Size(s); i++) { item = PySequence_GetItem(s, i); do_something_double(data, n, PyInt_AsSsize_t(item)); Py_DECREF(item); } } else goto error; Py_DECREF(array); Py_RETURN_NONE; error: Py_XDECREF(array); return NULL; }

2 1

is there a faster way to get a buffer interface than ndarray.tostring()?
by Chris Colbert Feb. 25, 2009

Feb. 25, 2009

Hi all, I'm new to mailing list and relatively new (~1 year) to python/numpy. I would appreciate any insight any of you may have here. The last 8 hours of digging through the docs has left me, finally, stuck. I am making a wxPython program that includes webcam functionality. The script below just brings up a simple display window that shows the webcam feed. The problem I am running into is in the WebCamWorker.run() method. This method reads the raw pixel data (in string form) from the webcam buffer. This data (thank you microsoft!) comes in BGR and bottom-to-top. I feed this buffer to the array constructor and then swap the pixel order to RGB and top-to-bottom. This all happens very fast, ~1ms on a Quad Core, and the majority of that time is spent constructing the array (the numpy pixel swapping is on the order of E-5s !). The tostring() method, however, takes a whopping 10ms to execute. Unfortunately, the wx.BitmapFromBuffer needs a single segment buffer interface as an argument. Is there a way to expose my numpy array as a buffer to cut down on this conversion time? I have tried sending the array to the PIL Image.fromarray(), but that eventually requires me to do a Image.tostring() anyway, which negates any benefit. Thanks in advance for the help! S. Chris Colbert Rehabilitation Robotics Laboratory University of South Florida #### Code ####### import VideoCapture import wx import time import threading import numpy as np import Image class WebCamWorker(threading.Thread): _abort = 0 def __init__(self, parent): super(WebCamWorker, self).__init__() self._parent = parent self.panel = wx.Panel(parent) def run(self): while not self._abort: #numpy arrays reverse pixel data that comes in from CCD as BGR and bottom to top pixeldata = np.ndarray((480,640,3), buffer=webcam.getBuffer()[0], dtype='u1') revpixels = pixeldata[::-1,:,::-1].tostring() #tostring is an order of magnitude slower than the entire array manipulation. need a faster method. rgbBMP = wx.BitmapFromBuffer(640, 480, revpixels) dc = wx.ClientDC(self.panel) dc.DrawBitmap(rgbBMP, 0, 0) #b = time.clock() #print b-a time.sleep(0.015) return class TestWxWebCam(wx.Frame): def __init__(self, parent): wx.Frame.__init__(self, parent, -1, size=(640,480)) self.worker = WebCamWorker(self) self.worker.start() class TestApp(wx.App): def OnInit(self): self.frame = TestWxWebCam(None) self.frame.Show() return True def OnExit(self): WebCamWorker._abort = 1 #this is a hack if __name__ == '__main__': webcam = VideoCapture.Device() webcam.setResolution(640, 480) webcam.displayCapturePinProperties() app = TestApp() app.MainLoop() del webcam ### /Code ######

4 6

Creating arrays with 'titles' in dtype causes TypeError on data access
by Ralph Heinkel Feb. 25, 2009

Feb. 25, 2009

Hi, I'm trying to use the additional 'title' feature of dtypes when creating arrays. The reason for using titles is that I want to store meta data about the columns in them (so do-not-use-titles is not the right solution for me...) Everything works fine without titles: >>> arr = array([('john', 4),('mark', 3)], dtype=[('name','O'),('id',int)]) >>> arr[0] ('john', 4) >>> arr[0][0] 'john' However here something is strange: >>> arr = array([('john', 4),('mark', 3)], dtype=[(('source:yy', 'name'),'O'),(('source:xx','id'),int)]) >>> arr[0] ('john', 4) >>> arr[0][0] Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: function takes at most 2 arguments (3 given) Any ideas what I'm doing wrong? Any help would be appreciated. Thanks, Ralph Using: - Python V 2.6 (but same error with python 2.5) - Numpy 1.2.1 (but same error on numpy 1.0.3.1) - opensuse 10.3

2 1

isbuiltin - failure of understanding
by Matthew Brett Feb. 24, 2009

Feb. 24, 2009

Hi, I was just trying to write a docstring for np.dtype.isbuiltin, when I realized I didn't understand it. As far as I can see, isbuitin should return: 0 for structured array dtypes 1 for types compiled into numpy 2 for extension types using the numpy C-API type extension machinery. Here's the C code: static PyObject * arraydescr_isbuiltin_get(PyArray_Descr *self) { long val; val = 0; if (self->fields == Py_None) val = 1; if (PyTypeNum_ISUSERDEF(self->type_num)) val = 2; return PyInt_FromLong(val); } But, why is this? In [2]: dt = np.dtype('S1') In [3]: dt Out[3]: dtype('|S1') In [4]: dt.isbuiltin Out[4]: 0 In [5]: print dt.fields None In [6]: print dt.fields == None True Same for np.dtype('U1') Best, Matthew

2 2

tensordot bug when summing over same axis indexes?
by Jose Borreguero Feb. 24, 2009

Feb. 24, 2009

The following example: from numpy import * a=arange(12).reshape(2,3,2) b=arange(24).reshape(2,3,2,2) c=tensordot( a,b,axes=([0,0],[1,1]) ) defaults: c=tensordot( a,b,axes=([0,0],[1,1]) ) File "/usr/lib/python2.4/site-packages/numpy/core/numeric.py", line 359, in tensordot raise ValueError, "shape-mismatch for sum" ValueError: shape-mismatch for sum Am I doing something really stupid, or is this a bug? -Jose

2 2

Summary of ticket 937
by Nils Wagner Feb. 24, 2009

Feb. 24, 2009

Hi all, The summary of ticket 937 is incomplete. It should be "Complex matrices and lstsq". http://projects.scipy.org/scipy/numpy/ticket/937 Nils

2 1

probably simple, reverse a reshape...
by John [H2O] Feb. 23, 2009

Feb. 23, 2009

Hello, this is probably one of those questions that is going to seem simple after reading responses... I'm trying to get the original array indices out of a row number from a reshaped array... shouldn't this be possible somehow? import numpy as np a = np.ones((3,5,10)) #for testing, I've done the following: a[0,:,:5] = np.eye(5,5)*1 a[1,:,:5] = np.eye(5,5)*10 a[2,:,:5] = np.eye(5,5)*20 #Ok, so I reshape the array: b = a.reshape(15,10) #now I want to be able to 'back out' the original shape indices from the row number of the new array. #I thought I could do something like this: d0,d1,d2 = a.shape new_d0 = row/d1 new_d1 = row/d0 #But it doesn't quite work... any suggestions as to what I am missing? -- View this message in context: http://www.nabble.com/probably-simple%2C-reverse-a-reshape...-tp22171584p22… Sent from the Numpy-discussion mailing list archive at Nabble.com.

1 1

block matrix and sums of blocks
by Xavier Gnata Feb. 23, 2009

Feb. 23, 2009

Hi, Let us consider one kN x kM array. What is the fastest way to sum each k x k square block of A and to put all these results into a NxM array B? For instance: If A = [112233 112233 223311 223311] then B = [4 8 12 4 12 4] No sanity checks on the arrays shapes are requiered. Only speed matters ;) Xavier

3 4