Mailman 3 February 2004 - NumPy-Discussion

PyArray_ContiguousFromObject & PyDECREF
by Michiel Jan Laurens de Hoon 17 May '11

17 May '11

Dear pythoneers, In Chapter 12 ("Writing a C extension") in the Numerical Python manual, in the first example PyArray_ContiguousFromObject is used together with PyDECREF, while in the last example PyArray_ContiguousFromObject is used but the arrays are not PyDECREF'ed before the function returns. From Chapter 13 ("C API Reference") it seems that a PyDECREF is needed in both cases. Or am I missing something here? Is PyDECREF not needed for some reason in the last example? Thanks in advance, Michiel de Hoon U Tokyo First example: ============== static PyObject * trace(PyObject *self, PyObject *args) { PyObject *input; PyArrayObject *array; double sum; int i, n; if (!PyArg_ParseTuple(args, "O", &input)) return NULL; array = (PyArrayObject *) PyArray_ContiguousFromObject(input, PyArray_DOUBLE, 2, 2); if (array == NULL) return NULL; n = array->dimensions[0]; if (n > array->dimensions[1]) n = array->dimensions[1]; sum = 0.; for (i = 0; i < n; i++) sum += *(double *)(array->data + i*array->strides[0] + i*array->strides[1]); Py_DECREF(array); return PyFloat_FromDouble(sum); } Last example: ============= static PyObject * matrix_vector(PyObject *self, PyObject *args) { PyObject *input1, *input2; PyArrayObject *matrix, *vector, *result; int dimensions[1]; double factor[1]; double real_zero[1] = {0.}; long int_one[1] = {1}; long dim0[1], dim1[1]; extern dgemv_(char *trans, long *m, long *n, double *alpha, double *a, long *lda, double *x, long *incx, double *beta, double *Y, long *incy); if (!PyArg_ParseTuple(args, "dOO", factor, &input1, &input2)) return NULL; matrix = (PyArrayObject *) PyArray_ContiguousFromObject(input1, PyArray_DOUBLE, 2, 2); if (matrix == NULL) return NULL; vector = (PyArrayObject *) PyArray_ContiguousFromObject(input2, PyArray_DOUBLE, 1, 1); if (vector == NULL) return NULL; if (matrix->dimensions[1] != vector->dimensions[0]) { PyErr_SetString(PyExc_ValueError, "array dimensions are not compatible"); return NULL; } dimensions[0] = matrix->dimensions[0]; result = (PyArrayObject *)PyArray_FromDims(1, dimensions, PyArray_DOUBLE); if (result == NULL) return NULL; dim0[0] = (long)matrix->dimensions[0]; dim1[0] = (long)matrix->dimensions[1]; dgemv_("T", dim1, dim0, factor, (double *)matrix->data, dim1, (double *)vector->data, int_one, real_zero, (double *)result->data, int_one); return PyArray_Return(result); } -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

3 2

Garbage collection fails after numarray exception
by Leo Breebaart 29 Feb '04

29 Feb '04

Hi all, I'm not sure if I have found a bug, or simply a type of behaviour that should not have surprised me -- but I most definitely did not expect the following numarray 0.8 behaviour: >>> import numarray >>> a = numarray.zeros(100000000) >>> del a >>> a = numarray.zeros(100000000) >>> a + '' Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib/python2.3/site-packages/numarray/numarraycore.py", line 664, in __add__ return ufunc.add(self, operand) File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 870, in _cache_miss2 (in1, in2), insig, scalar = _inputcheck(n1, n2) File "/usr/lib/python2.3/site-packages/numarray/ufunc.py", line 391, in _inputcheck raise TypeError( TypeError: UFunc arguments must be numarray, scalars or numeric sequences >>> del a >>> If I execute these statements alongside 'top' or another memory monitor, I of course see a big memory increase after each call to 'zeros', as well as a big decrease after the first 'del a' -- but no change whatsoever any more after the second 'del a', not even if I explicitly call the garbage collector via the gc module. As far as I can tell, the occurrence of the exception somehow causes a permanent increase in a's refcount, with a big memory leak as a result. (In case anyone's curious about the context for this, the """ a + '' """" construct was taken straight from the Python Cookbook, as a way of determining whether an object exhibits string-like behaviour (useful for printing methods, etc.)) I understand that I can easily work around this problem so that numarray objects will not be fed to this construct to begin with, but I do wonder if it is really true (and intended) that numarray exceptions have the side effect of making certain potentially huge objects indestructible. Can anyone here shed some light on this? Many thanks in advance, -- Leo Breebaart <leo(a)lspace.org>

3 5

numarray, BLAS, LAPACK and MacOS X
by Luc Bourhis 26 Feb '04

26 Feb '04

MacOS X (at least 10.2 and 10.3) comes with BLAS and LAPACK already installed. However the hooks in addons.py (i.e. setting the environment variable USE_LAPACK and then filling the lists lapack_libs and lapack_dirs) don't work because it will typically pass the arguments -llapack, -lblas to the linker although Apple does not supply any liblapack.a or libblas.a. Instead Apple uses the standard MacOS X distribution scheme: a framework (vecLib is the name). So the correct way to compile numarray with the native blas and lapack on MacOS X is to fix the addons.py as follow: Extension('numarray.linear_algebra.lapack_lite2', sourcelist, include_dirs = ["Packages/LinearAlgebra2/Src", 'Include/numarray'], extra_objects = ["-bundle", "-framework", "vecLib",]), I checked it worked (but the -bundle may not be necessary) HtH somebody, -- Luc Bourhis Chemical Crystallography Laboratory University of Durham

2 1

"GC object already in linked list" error when testall.test()
by Tina Li 25 Feb '04

25 Feb '04

Hello, I installed numarray on a RedHat 8.0 with Python 2.2.1. The build completed fine, but it failed when I tried to testall.test(). Here is the output: ------------------------------------------------------------ Python 2.2.1 (#1, Aug 30 2002, 12:15:30) [GCC 3.2 20020822 (Red Hat Linux Rawhide 3.2-4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import numarray.testall as testall >>> testall.test() numarray: ((0, 1142), (0, 1142)) numarray.records: (0, 48) numarray.strings: (0, 168) numarray.memmap: (0, 82) numarray.objects: (0, 68) numarray.memorytest: (0, 16) Fatal Python error: GC object already in linked list Aborted ------------------------------------------------------------ Seems like a garbage collector problem. That's all I can tell. I installed using "python setup.py install --gencode --prefix=<installation directory>" and then "export PYTHONPATH". Does anyone have a clue how this can be fixed? Thanks! Tina

2 2

RE: NA_updateDataPtr segmentation fault bug?
by Marco Bubke 25 Feb '04

25 Feb '04

Ok cdef NumArray array_to_float(NumArray array): cdef NumArray new_array new_array = array NA_updateDataPtr(new_array) NA_updateDataPtr(new_array) flat_array = NA_InputArray(array, tFloat32, NUM_C_ARRAY) return flat_array Here the C code: static PyArrayObject *__pyx_f_2gl_array_to_float(PyArrayObject *__pyx_v_array) { PyArrayObject *__pyx_v_new_array; PyObject *__pyx_v_flat_array; PyArrayObject *__pyx_r; PyObject *__pyx_1 = 0; Py_INCREF(__pyx_v_array); ((PyObject*)__pyx_v_new_array) = Py_None; Py_INCREF(((PyObject*)__pyx_v_new_array)); __pyx_v_flat_array = Py_None; Py_INCREF(__pyx_v_flat_array); /* "/home/marco/projects/calliopeia/numarray.pxd":102 */ Py_INCREF(((PyObject *)__pyx_v_array)); Py_DECREF(((PyObject *)__pyx_v_new_array)); ((PyObject *)__pyx_v_new_array) = ((PyObject *)__pyx_v_array); /* "/home/marco/projects/calliopeia/numarray.pxd":103 */ __pyx_1 = NA_updateDataPtr(((PyObject *)__pyx_v_new_array)); if (!__pyx_1) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 103; goto __pyx_L 1;} Py_DECREF(__pyx_1); __pyx_1 = 0; /* "/home/marco/projects/calliopeia/numarray.pxd":104 */ __pyx_1 = NA_updateDataPtr(((PyObject *)__pyx_v_new_array)); if (!__pyx_1) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 104; goto __pyx_L 1;} Py_DECREF(__pyx_1); __pyx_1 = 0; /* "/home/marco/projects/calliopeia/numarray.pxd":105 */ __pyx_1 = NA_InputArray(((PyObject *)__pyx_v_array),tFloat32,NUM_C_ARRAY); if (!__pyx_1) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 105 ; goto __pyx_L1;} Py_DECREF(__pyx_v_flat_array); __pyx_v_flat_array = __pyx_1; __pyx_1 = 0; /* "/home/marco/projects/calliopeia/numarray.pxd":106 */ if (!__Pyx_TypeTest(__pyx_v_flat_array, __pyx_ptype_2gl__numarray)) {__pyx_filename = __pyx_f[0]; __pyx_lineno = 106; goto __pyx_L1;} Py_INCREF(__pyx_v_flat_array); __pyx_r = (PyArrayObject *)__pyx_v_flat_array; goto __pyx_L0; (PyObject *)__pyx_r = Py_None; Py_INCREF((PyObject *)__pyx_r); goto __pyx_L0; __pyx_L1:; Py_XDECREF(__pyx_1); __Pyx_AddTraceback("gl.array_to_float"); __pyx_r = 0; __pyx_L0:; Py_DECREF(__pyx_v_new_array); Py_DECREF(__pyx_v_flat_array); Py_DECREF(__pyx_v_array); return ((PyArrayObject *)__pyx_r) ; }

2 1

Deleting arrays.
by Andrea Riciputi 25 Feb '04

25 Feb '04

Hi, I'm running a simulation program partly written in C partly in Python and I use Numeric arrays extensively. At the end of a single run I have a bunch of Numeric arrays containing partial results that I don't need anymore. These arrays have been created on the C side of my code and then copied (actually linked via PyArray_FromDimsAndData) to Numeric arrays. How can I release these arrays and get back memory in order to be able to start the next run? Thanks in advance, Andrea. --- Andrea Riciputi "Science is like sex: sometimes something useful comes out, but that is not the reason we are doing it" -- (Richard Feynman)

3 5

NA_updateDataPtr segmentation fault bug?
by Marco Bubke 25 Feb '04

25 Feb '04

Hi I have a little function in pyrex: cdef NumArray array_to_float(NumArray array): # maybe here is memoty leak! cdef NumArray flat_array cdef NumArray new_array print 'mark 1' new_array = array.astype('Float32') print new_array print 'mark 2' NA_updateDataPtr(new_array) print 'mark 3' NA_updateDataPtr(new_array) print 'mark 4' flat_array = NA_InputArray(new_array, tFloat32, NUM_C_ARRAY) print 'mark 5' return flat_array After the second NA_updateDataPtr I get a segmentation fault> Otherwise I get it with the NA_InputArray. It goes down to getReadBufferDataPtr and than to rval = buff2->ob_type->tp_as_buffer->bf_getreadbuffer(buff2, 0, buff); I believe bf_getreadbuffer is memory_getbuf but I'm not sure. I don't the bug but its happen. Can somebody help me? thanks and regards Marco

2 1

User arrays in numarray
by Antti Korvenoja 25 Feb '04

25 Feb '04

Dear numarray developers, What would be the recommended class to inherit from if one wants to derive a new class from numarray array? In other words, what would be similar to UserArray in Numeric (NDArray, NumArray)? -Antti Korvenoja

2 1

PROTECT Your PRIVACY, Block Annoying Pop-ups, Cut Down Employee Waste, Tue, 24 Feb 2004 07:25:22 -0500
by Marion Hubbard 24 Feb '04

24 Feb '04

Tue, 24 Feb 2004 07:25:22 -0500 PRODUCT PARADISE - Protect Your Privacy - Cut Down On Employee Waste - Block Annoying Pop-ups - All with state-of-the-art, affordable software from Product Paradise. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Safe Surfer Pro NEW v2.0 Safe Surfer Pro protects your privacy by erasing your Internet records from your computer and is simply the best internet eraser available! Not only can it clean your internet tracks, but it will erase your internet tracks as you surf. Feel more secure and clean as you surf. Was 49.95 - Now just 39.95. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Secret Server Pro Would you like to find out if your kids are browsing the web where they shouldn't be? Or do you need to know if your employees are wasting your time? This product also enables you to record instant messaging communication done from your computer. Now just 39.95. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Popup Killer Popup Killer blocks annoying popups that slow down and destroy your surfing experience. Popup Killer installs in seconds and blocks all popups with an easy to use and customizable interface. By constantly eliminating popups, you will enjoy a faster web experience and not have to continuously close those annoying extra windows. Just 29.95. Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 Or GET THE BEST DEAL 3 programs for the price of less than 2 - Only 74.95 Enter Paradise Now: http://click.com-click.com.ph/click.php?id=prcoomi&ID=11 No more ads: http://www.productparadise.co.om/nomoreads.html blumenthal bulb meditate repertory dayton luxuriate expect honesty maintain addressee spacious industry lavabo acts asunder siegel rotor forborne gina anaglyph benign buses tagging malocclusion classroom colossal deteriorate odorous clot thunderous amadeus

1 0

ANN: F2PY - Fortran to Python Interface Generator
by Pearu Peterson 24 Feb '04

24 Feb '04

F2PY - Fortran to Python Interface Generator -------------------------------------------- I am pleased to announce the seventh public release of F2PY, version 2.39.235_1642. The purpose of the F2PY project is to provide the connection between Python and Fortran programming languages. For more information, see http://cens.ioc.ee/projects/f2py2e/ Download source: http://cens.ioc.ee/projects/f2py2e/2.x/F2PY-2-latest.tar.gz What's new? ------------ Since the last public release, that was more than two years ago, F2PY project has been actively maintained and many useful features have been implemented. Here follows some highlights: * New statement ``usercode`` allows inserting user defined C code to F2PY generated extension module sources at various relevant places: before wrapper functions, after variable declarations, and to the end of initialization function of the module. This makes using F2PY very flexible. * New statement ``pymethoddef`` allows adding items to PyMethodDef-array. * Full support for character arrays and arrays of strings is finally implemented. * Number of feature requests from users are implemented. For example: - the auxiliary ``as_column_major_storage()`` that efficiently converts array to column storage order - the F2PY_REPORT_ON_ARRAY_COPY macro that when defined sends a message to stderr when copy of an array with the size larger than specified threshold is made. - Numarray support, thanks to Todd Miller. * Support for Win32 and Mac OSX platforms is considerably improved. The list of compilers supported by the scipy_distutils package is longer than ever: GNU Fortran Compiler Portland Group Fortran Compiler Absoft Corp Fortran Compiler MIPSpro Fortran Compiler Sun|Forte Fortran 95 Compiler Intel Fortran Compiler for 32-bit apps Intel Visual Fortran Compiler for 32-bit apps Intel Fortran Compiler for Itanium apps NAGWare Fortran 95 Compiler Compaq Fortran Compiler DIGITAL|Compaq Visual Fortran Compiler Pacific-Sierra Research Fortran 90 Compiler HP Fortran 90 Compiler Lahey/Fujitsu Fortran 95 Compiler IBM XL Fortran Compiler * Numerous bugs are fixed. You should definitely update F2PY when using complex input arrays, there was a nasty bug that in certain cases caused incorrect results. * F2PY has now a man page and its documentation is kept up to date. Many other improvements to F2PY algorithm and usage are implemented, see HISTORY.txt for more details. Enjoy, Pearu Peterson --------------- <P><A HREF="http://cens.ioc.ee/projects/f2py2e/">F2PY 2.39.235_1642</A> - The Fortran to Python Interface Generator (24-Feb-04)

1 0