Mailman 3 August 2007 - NumPy-Discussion

PyArray_ContiguousFromObject & PyDECREF
by Michiel Jan Laurens de Hoon 17 May '11

17 May '11

Dear pythoneers, In Chapter 12 ("Writing a C extension") in the Numerical Python manual, in the first example PyArray_ContiguousFromObject is used together with PyDECREF, while in the last example PyArray_ContiguousFromObject is used but the arrays are not PyDECREF'ed before the function returns. From Chapter 13 ("C API Reference") it seems that a PyDECREF is needed in both cases. Or am I missing something here? Is PyDECREF not needed for some reason in the last example? Thanks in advance, Michiel de Hoon U Tokyo First example: ============== static PyObject * trace(PyObject *self, PyObject *args) { PyObject *input; PyArrayObject *array; double sum; int i, n; if (!PyArg_ParseTuple(args, "O", &input)) return NULL; array = (PyArrayObject *) PyArray_ContiguousFromObject(input, PyArray_DOUBLE, 2, 2); if (array == NULL) return NULL; n = array->dimensions[0]; if (n > array->dimensions[1]) n = array->dimensions[1]; sum = 0.; for (i = 0; i < n; i++) sum += *(double *)(array->data + i*array->strides[0] + i*array->strides[1]); Py_DECREF(array); return PyFloat_FromDouble(sum); } Last example: ============= static PyObject * matrix_vector(PyObject *self, PyObject *args) { PyObject *input1, *input2; PyArrayObject *matrix, *vector, *result; int dimensions[1]; double factor[1]; double real_zero[1] = {0.}; long int_one[1] = {1}; long dim0[1], dim1[1]; extern dgemv_(char *trans, long *m, long *n, double *alpha, double *a, long *lda, double *x, long *incx, double *beta, double *Y, long *incy); if (!PyArg_ParseTuple(args, "dOO", factor, &input1, &input2)) return NULL; matrix = (PyArrayObject *) PyArray_ContiguousFromObject(input1, PyArray_DOUBLE, 2, 2); if (matrix == NULL) return NULL; vector = (PyArrayObject *) PyArray_ContiguousFromObject(input2, PyArray_DOUBLE, 1, 1); if (vector == NULL) return NULL; if (matrix->dimensions[1] != vector->dimensions[0]) { PyErr_SetString(PyExc_ValueError, "array dimensions are not compatible"); return NULL; } dimensions[0] = matrix->dimensions[0]; result = (PyArrayObject *)PyArray_FromDims(1, dimensions, PyArray_DOUBLE); if (result == NULL) return NULL; dim0[0] = (long)matrix->dimensions[0]; dim1[0] = (long)matrix->dimensions[1]; dgemv_("T", dim1, dim0, factor, (double *)matrix->data, dim1, (double *)vector->data, int_one, real_zero, (double *)result->data, int_one); return PyArray_Return(result); } -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

3 2

Unpleasant behavior with poly1d and numpy scalar multiplication
by Fernando Perez 15 Feb '10

15 Feb '10

Hi all, consider this little script: from numpy import poly1d, float, float32 p=poly1d([1.,2.]) three=float(3) three32=float32(3) print 'three*p:',three*p print 'three32*p:',three32*p print 'p*three32:',p*three32 which produces when run: In [3]: run pol1d.py three*p: 3 x + 6 three32*p: [ 3. 6.] p*three32: 3 x + 6 The fact that multiplication between poly1d objects and numbers is: - non-commutative when the numbers are numpy scalars - different for the same number if it is a python float vs a numpy scalar is rather unpleasant, and I can see this causing hard to find bugs, depending on whether your code gets a parameter that came as a python float or a numpy one. This was found today by a colleague on numpy 1.0.4.dev3937. It feels like a bug to me, do others agree? Or is it consistent with a part of the zen of numpy I've missed thus far? Thanks, f

6 17

Compact way of performing array math with specified result type?
by Russell E. Owen 08 Feb '10

08 Feb '10

I often find myself doing simple math on sequences of numbers (which might or might not be numpy arrays) where I want the result (and thus the inputs) coerced to a particular data type. I'd like to be able to say: numpy.divide(seq1, seq2, dtype=float) but ufuncs don't allow on to specify a result type. So I do this instead: numpy.array(seq1, dtype=float) / numpy.array(seq2, dtype=float) Is there a more compact solution (without having to create the result array first and supply it as an argument)? -- Russell

4 3

Re: [Numpy-discussion] Floating point exception with numpy and embedded python interpreter
by Andrew Straw 20 Sep '07

20 Sep '07

Arkaitz Bitorika wrote: > > On 19 Apr 2006, at 18:37, Andrew Straw wrote: > >> >>> I've been embedding Python for ages on the same program and other >>> modules work fine, only numpy fails. >> >> >> Most other modules don't use the SSE units, so wouldn't get hit by such >> a bug. > > > Is there a way of not using those units from numpy, to check if > that's what's going on? I think that numpy only accesses the SSE units through ATLAS or other external library. So, build numpy without ATLAS. But I'm not 100% sure anymore if there aren't any optimizations that directly use SSE if it's available. > Or alternatively, how would I check if my program is messing with the > SSE bits? Hmm, I think that's a bit hairy. I'd suggest simply asking the C++ library's mailing list if they alter the error bits on the control registers of the SSE unit. (Out of curiousity, what library is it?) If you want hairy, though, I think you'd have to check from C with the appropriate calls -- I'd start with the source code in that bug report. It looks like they're inlining an assembly statement to query a SSE control register.

5 7

"Extended" Outer Product
by Geoffrey Zhu 20 Sep '07

20 Sep '07

Hi Everyone, I am wondering if there is an "extended" outer product. Take the example in "Guide to Numpy." Instead of doing an multiplication, I want to call a custom function for each pair. >>> print outer([1,2,3],[10,100,1000]) [[ 10 100 1000] [ 20 200 2000] [ 30 300 3000]] So I want: [ [f(1,10), f(1,100), f(1,1000)], [f(2,10), f(2, 100), f(2, 1000)], [f(3,10), f(3, 100), f(3,1000)] ] Does anyone know how to do this without using a double loop? Thanks, Geoffrey

8 13

latest svn version fails on Solaris
by Christopher Hanley 20 Sep '07

20 Sep '07

Hi, The latest version of numpy has a unit test failure on big endian machines. ====================================================================== FAIL: test_record_array (numpy.core.tests.test_multiarray.test_putmask) ---------------------------------------------------------------------- Traceback (most recent call last): File "/data/basil5/site-packages/lib/python/numpy/core/tests/test_multiarray.py", line 450, in test_record_array assert_array_equal(rec['x'],[10,5]) File "/data/basil5/site-packages/lib/python/numpy/testing/utils.py", line 223, in assert_array_equal verbose=verbose, header='Arrays are not equal') File "/data/basil5/site-packages/lib/python/numpy/testing/utils.py", line 215, in assert_array_compare assert cond, msg AssertionError: Arrays are not equal (mismatch 50.0%) x: array([ 4.58492919e-320, 5.00000000e+000]) y: array([10, 5]) ---------------------------------------------------------------------- Ran 670 tests in 47.182s Chris

3 2

Building numpy 1.0.3-2 on Linux 2.6.8 i686 (Debian 3.1)
by Michael Hoffman 15 Sep '07

15 Sep '07

Hi. I have been trying to build NumPy on a 32-bit Linux box using python setup.py build. I received the following errors: """ creating build/temp.linux-i686-2.5/numpy/linalg compile options: '-DATLAS_WITHOUT_LAPACK -DATLAS_INFO="\"3.6.0\"" -I/usr/include -Inumpy/core/include -Ibuild/src.linux-i686-2.5/numpy/core -I/nfs/acari/mh5/include/python2.5 -I/nfs/acari/mh5/include -I/software/python-2.5/include/python2.5 -Inumpy/core/src -Inumpy/core/include -c' gcc: numpy/linalg/lapack_litemodule.c g77 build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o -L/usr/lib/sse2 -L/usr/lib -lf77blas -lcblas -latlas -llapack -lg2c-pic -o build/lib.linux-i686-2.5/numpy/linalg/lapack_lite.so build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0x44): In function `initlapack_lite': numpy/linalg/lapack_litemodule.c:827: undefined reference to `Py_InitModule4' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0x57):numpy/linalg/lapack_litemodule.c:831: undefined reference to `PyModule_GetDict' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0x73):numpy/linalg/lapack_litemodule.c:832: undefined reference to `PyErr_NewException' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0x8f):numpy/linalg/lapack_litemodule.c:833: undefined reference to `PyDict_SetItemString' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0xa1):numpy/linalg/lapack_litemodule.c:836: undefined reference to `PyErr_Print' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0xa7):numpy/linalg/lapack_litemodule.c:836: undefined reference to `PyExc_ImportError' build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o(.text+0xbb):numpy/linalg/lapack_litemodule.c:836: undefined reference to `PyErr_SetString' [...] /usr/lib/libfrtbegin.a(frtbegin.o)(.text+0x32): In function `main': : undefined reference to `MAIN__' collect2: ld returned 1 exit status error: Command "g77 build/temp.linux-i686-2.5/numpy/linalg/lapack_litemodule.o -L/usr/lib/sse2 -L/usr/lib -lf77blas -lcblas -latlas -llapack -lg2c-pic -o build/lib.linux-i686-2.5/numpy/linalg/lapack_lite.so" failed with exit status 1 """ After some Googling, I found a recommendation to unset CFLAGS and LDFLAGS, but they are not set in the transcript above. Any suggestions? -- Michael Hoffman

2 5

numpy build fails on powerpc ydl
by Vincent Broman 31 Aug '07

31 Aug '07

My build of numpy fails under Yellow Dog Linux 2.1, running on a powerpc multiprocessor board from Curtiss-Wright. Its kernel is 2.4.19-Asmp tailored by the vendor. The gcc compiler is configured as ppc-yellowdog-linux with version number 2.95.3 20010111. The python I'm using is Python 2.5.1 (r251:54863) installed as python2. Plain /usr/bin/python is 1.5.x . The numpy version I'm trying to build is r4003 for v1.0.4 . The setup fails compiling build/src.linux-ppc-2.5/numpy/core/src/umathmodule.c with a long list of error messages of the following two kinds. warning: conflicting types for built-in function `sinl' repeated for `cosl', `fabsl', and `sqrtl', triggered by line 442. inconsistent operand constraints in an `asm', triggered by lines 1100, 1124, 1150, 1755, 1785, and 1834. I cannot see on those source lines what causes such a message; I suspect there is some long complicated cpp macro or asm statement in some include file which I don't find. Has anyone tried building numpy on Yellow Dog Linux or on a PowerPC with gcc? Vincent Broman broman(a)spawar.navy.mil

3 2

Error code of NumpyTest()
by Matthieu Brucher 30 Aug '07

30 Aug '07

Hi, I wondered if there was a way of returning another error code than 0 when executing the test suite so that a parent process can immediately know if all the tests passed or not. The numpy buildbot seems to have the same behaviour BTW. I don't know if it is possible, but it would be great. Matthieu

3 3

Accessing a numpy array in a mmap fashion
by Brian Donovan 30 Aug '07

30 Aug '07

Hello all, I'm wondering if there is a way to use a numpy array that uses disk as a memory store rather than ram. I'm looking for something like mmap but which can be used like a numpy array. The general idea is this. I'm simulating a system which produces a large dataset over a few hours of processing time. Rather than store the numpy array in memory during processing I'd like to write the data directly to disk but still be able to treat the array as a numpy array. Is this possible? Any ideas? Thanks, Brian -- Brian Donovan Research Assistant Microwave Remote Sensing Lab UMass Amherst

3 2