Mailman 3 July 2007 - NumPy-Discussion

PyArray_ContiguousFromObject & PyDECREF
by Michiel Jan Laurens de Hoon May 17, 2011

May 17, 2011

Dear pythoneers, In Chapter 12 ("Writing a C extension") in the Numerical Python manual, in the first example PyArray_ContiguousFromObject is used together with PyDECREF, while in the last example PyArray_ContiguousFromObject is used but the arrays are not PyDECREF'ed before the function returns. From Chapter 13 ("C API Reference") it seems that a PyDECREF is needed in both cases. Or am I missing something here? Is PyDECREF not needed for some reason in the last example? Thanks in advance, Michiel de Hoon U Tokyo First example: ============== static PyObject * trace(PyObject *self, PyObject *args) { PyObject *input; PyArrayObject *array; double sum; int i, n; if (!PyArg_ParseTuple(args, "O", &input)) return NULL; array = (PyArrayObject *) PyArray_ContiguousFromObject(input, PyArray_DOUBLE, 2, 2); if (array == NULL) return NULL; n = array->dimensions[0]; if (n > array->dimensions[1]) n = array->dimensions[1]; sum = 0.; for (i = 0; i < n; i++) sum += *(double *)(array->data + i*array->strides[0] + i*array->strides[1]); Py_DECREF(array); return PyFloat_FromDouble(sum); } Last example: ============= static PyObject * matrix_vector(PyObject *self, PyObject *args) { PyObject *input1, *input2; PyArrayObject *matrix, *vector, *result; int dimensions[1]; double factor[1]; double real_zero[1] = {0.}; long int_one[1] = {1}; long dim0[1], dim1[1]; extern dgemv_(char *trans, long *m, long *n, double *alpha, double *a, long *lda, double *x, long *incx, double *beta, double *Y, long *incy); if (!PyArg_ParseTuple(args, "dOO", factor, &input1, &input2)) return NULL; matrix = (PyArrayObject *) PyArray_ContiguousFromObject(input1, PyArray_DOUBLE, 2, 2); if (matrix == NULL) return NULL; vector = (PyArrayObject *) PyArray_ContiguousFromObject(input2, PyArray_DOUBLE, 1, 1); if (vector == NULL) return NULL; if (matrix->dimensions[1] != vector->dimensions[0]) { PyErr_SetString(PyExc_ValueError, "array dimensions are not compatible"); return NULL; } dimensions[0] = matrix->dimensions[0]; result = (PyArrayObject *)PyArray_FromDims(1, dimensions, PyArray_DOUBLE); if (result == NULL) return NULL; dim0[0] = (long)matrix->dimensions[0]; dim1[0] = (long)matrix->dimensions[1]; dgemv_("T", dim1, dim0, factor, (double *)matrix->data, dim1, (double *)vector->data, int_one, real_zero, (double *)result->data, int_one); return PyArray_Return(result); } -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

3 2

Unpleasant behavior with poly1d and numpy scalar multiplication
by Fernando Perez Feb. 15, 2010

Feb. 15, 2010

Hi all, consider this little script: from numpy import poly1d, float, float32 p=poly1d([1.,2.]) three=float(3) three32=float32(3) print 'three*p:',three*p print 'three32*p:',three32*p print 'p*three32:',p*three32 which produces when run: In [3]: run pol1d.py three*p: 3 x + 6 three32*p: [ 3. 6.] p*three32: 3 x + 6 The fact that multiplication between poly1d objects and numbers is: - non-commutative when the numbers are numpy scalars - different for the same number if it is a python float vs a numpy scalar is rather unpleasant, and I can see this causing hard to find bugs, depending on whether your code gets a parameter that came as a python float or a numpy one. This was found today by a colleague on numpy 1.0.4.dev3937. It feels like a bug to me, do others agree? Or is it consistent with a part of the zen of numpy I've missed thus far? Thanks, f

6 17

Compact way of performing array math with specified result type?
by Russell E. Owen Feb. 8, 2010

Feb. 8, 2010

I often find myself doing simple math on sequences of numbers (which might or might not be numpy arrays) where I want the result (and thus the inputs) coerced to a particular data type. I'd like to be able to say: numpy.divide(seq1, seq2, dtype=float) but ufuncs don't allow on to specify a result type. So I do this instead: numpy.array(seq1, dtype=float) / numpy.array(seq2, dtype=float) Is there a more compact solution (without having to create the result array first and supply it as an argument)? -- Russell

4 3

Re: [Numpy-discussion] Floating point exception with numpy and embedded python interpreter
by Andrew Straw Sept. 20, 2007

Sept. 20, 2007

Arkaitz Bitorika wrote: > > On 19 Apr 2006, at 18:37, Andrew Straw wrote: > >> >>> I've been embedding Python for ages on the same program and other >>> modules work fine, only numpy fails. >> >> >> Most other modules don't use the SSE units, so wouldn't get hit by such >> a bug. > > > Is there a way of not using those units from numpy, to check if > that's what's going on? I think that numpy only accesses the SSE units through ATLAS or other external library. So, build numpy without ATLAS. But I'm not 100% sure anymore if there aren't any optimizations that directly use SSE if it's available. > Or alternatively, how would I check if my program is messing with the > SSE bits? Hmm, I think that's a bit hairy. I'd suggest simply asking the C++ library's mailing list if they alter the error bits on the control registers of the SSE unit. (Out of curiousity, what library is it?) If you want hairy, though, I think you'd have to check from C with the appropriate calls -- I'd start with the source code in that bug report. It looks like they're inlining an assembly statement to query a SSE control register.

5 7

Building numpy 1.0.3-2 on Linux 2.6.8 i686 (Debian 3.1)
by Michael Hoffman Sept. 15, 2007

Sept. 15, 2007

Hi. I have been trying to build NumPy on a 32-bit Linux box using python setup.py build. I received the following errors: """ creating build/temp.linux-i686-2.5/numpy/linalg compile options: '-DATLAS_WITHOUT_LAPACK -DATLAS_INFO="\"3.6.0\"" -I/usr/include -Inumpy/core/include -Ibuild/src.linux-i686-2.5/numpy/core -I/nfs/acari/mh5/include/python2.5 -I/nfs/acari/mh5/include -I/software/python-2.5/include/python2.5 -Inumpy/core/src -Inumpy/core/include -c' gcc: numpy/linalg/lapack_litemodule.c g77 build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o -L/usr/lib/sse2 -L/usr/lib -lf77blas -lcblas -latlas -llapack -lg2c-pic -o build/lib.linux-i686-2.5/numpy/linalg/lapack_lite.so build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0x44): In function `initlapack_lite': numpy/linalg/lapack_litemodule.c:827: undefined reference to `Py_InitModule4' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0x57):numpy/linalg/lapack_litemodule.c:831: undefined reference to `PyModule_GetDict' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0x73):numpy/linalg/lapack_litemodule.c:832: undefined reference to `PyErr_NewException' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0x8f):numpy/linalg/lapack_litemodule.c:833: undefined reference to `PyDict_SetItemString' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0xa1):numpy/linalg/lapack_litemodule.c:836: undefined reference to `PyErr_Print' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0xa7):numpy/linalg/lapack_litemodule.c:836: undefined reference to `PyExc_ImportError' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0xbb):numpy/linalg/lapack_litemodule.c:836: undefined reference to `PyErr_SetString' [...] /usr/lib/libfrtbegin.a(frtbegin.o)(.text+0x32): In function `main': : undefined reference to `MAIN__' collect2: ld returned 1 exit status error: Command "g77 build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o -L/usr/lib/sse2 -L/usr/lib -lf77blas -lcblas -latlas -llapack -lg2c-pic -o build/lib.linux-i686-2.5/numpy/linalg/lapack_lite.so" failed with exit status 1 """ After some Googling, I found a recommendation to unset CFLAGS and LDFLAGS, but they are not set in the transcript above. Any suggestions? -- Michael Hoffman

2 5

.transpose() of memmap array fails to close()
by Glen W. Mabey Aug. 17, 2007

Aug. 17, 2007

Hello, When assigning a variable that is the transpose() of a memmap array, the ._mmap member doesn't get copied, I guess: In [1]:import numpy In [2]:amemmap = numpy.memmap( '/tmp/afile', dtype=numpy.float32, shape=(4,5), mode='w+' ) In [3]:bmemmap = amemmap.transpose() In [4]:bmemmap.close() --------------------------------------------------------------------------- <type 'exceptions.AttributeError'> Traceback (most recent call last) /home/gmabey/src/R9619_dev_acqlibweb/Projects/R9619_NChannelDetection/NED/<ipython console> in <module>() /usr/local/stow/numpy-20070605_svn-py2.5/lib/python2.5/site-packages/numpy/core/memmap.py in close(self) 86 87 def close(self): ---> 88 self._mmap.close() 89 90 def __del__(self): <type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'close' > /usr/local/stow/numpy-20070605_svn-py2.5/lib/python2.5/site-packages/numpy/core/memmap.py(88)close() 87 def close(self): ---> 88 self._mmap.close() 89 This is an issue when the data is accessed in an order that is different from how it is stored on disk, as: bmemmap = numpy.memmap( '/tmp/afile', dtype=numpy.float32, shape=(4,5), mode='w+' ).transpose() So the object that was originally produced not accessible. I imagine there is some better way to indicate order of dimensions, but regardless, doing In [4]:bmemmap._mmap = amemmap._mmap is a hack workaround. Best regards, Glen Mabey

4 11

rant against from numpy import * / from pylab import *
by Sebastian Haase Aug. 9, 2007

Aug. 9, 2007

Hi! I use the wxPython PyShell. I like especially the feature that when typing a module and then the dot "." I get a popup list of all available functions (names) inside that module. Secondly, I think it really makes code clearer when one can see where a function comes from. I have a default import numpy as N executed before my shell even starts. In fact I have a bunch of my "standard" modules imported as <some single capital letter>. This - I think - is a good compromise to the commonly used "extra typing" and "unreadable" argument. a = sin(b) * arange(10,50, .1) * cos(d) vs. a = N.sin(b) * N.arange(10,50, .1) * N.cos(d) I would like to hear some comments by others. On a different note: I just started using pylab, so I did added an automatic "from matplotlib import pylab as P" -- but now P contains everything that I already have in N. It makes it really hard to *find* (as in *see* n the popup-list) the pylab-only functions. -- what can I do about this ? Thanks, Sebastian

2 3

numpy installation problem
by kingshuk ghosh Aug. 8, 2007

Aug. 8, 2007

Hi, I downloaded numpy1.0.3-2.tar and unzipped and untared. However somehow new numpy does not work. It invokes the old numpy 0.9.6 when i import numpy from python and type in numpy.version.version . I tried to change path and once I do that and when I do import numpy it says "running from source directory" and then if I try numpy.version.version it gives some error. Is there something obvious I am missing after unzipping and untaring the numpy source file ? For example do I need to do something to install the new numpy1.0.3 ? Or do I also need to download full python package ? I am trying to run this on Red Hat Linux 3.2.2-5 which has a gcc 3.2.2 and the version of python is 2.4 . Any help will be greatly appreciated. Cheers Kings

2 1

How to implement a 'pivot table?'
by Geoffrey Zhu Aug. 6, 2007

Aug. 6, 2007

Hi Everyone, I am wondering what is the best (and fast) way to build a pivot table aside from the 'brute force way?' I want to transform an numpy array into a pivot table. For example, if I have a numpy array like below: Region Date # of Units ---------- ---------- -------------- East 1/1 10 East 1/1 20 East 1/2 30 West 1/1 40 West 1/2 50 West 1/2 60 I want to transform this into the following table, where f() is a given aggregate function: Date Region 1/1 1/2 ---------- East f(10,20) f(30) West f(40) f(50,60) I can regroup them into 'sets' and do it the brute force way, but that is kind of slow to execute. Does anyone know a better way? Thanks, Geoffrey

5 10

reading 10 bit raw data into an array
by Danny Chan Aug. 1, 2007

Aug. 1, 2007

Hi all! I'm trying to read a data file that contains a raw image file. Every pixel is assigned a value from 0 to 1023, and all pixels are stored from top left to bottom right pixel in binary format in this file. I know the width and the height of the image, so all that would be required is to read 10 bits at a time and store it these as an integer. I played around with the fromstring and fromfile function, and I read the documentation for dtype objects, but I'm still confused. It seems simple enough to read data in a format with a standard bitwidth, but how can I read data in a non-standard format. Can anyone help? Greets, Danny --------------------------------- Alles was der Gesundheit und Entspannung dient.BE A BETTER MEDIZINMANN!

4 5