Mailman 3 August 2004 - NumPy-Discussion

PyArray_ContiguousFromObject & PyDECREF
by Michiel Jan Laurens de Hoon May 17, 2011

May 17, 2011

Dear pythoneers, In Chapter 12 ("Writing a C extension") in the Numerical Python manual, in the first example PyArray_ContiguousFromObject is used together with PyDECREF, while in the last example PyArray_ContiguousFromObject is used but the arrays are not PyDECREF'ed before the function returns. From Chapter 13 ("C API Reference") it seems that a PyDECREF is needed in both cases. Or am I missing something here? Is PyDECREF not needed for some reason in the last example? Thanks in advance, Michiel de Hoon U Tokyo First example: ============== static PyObject * trace(PyObject *self, PyObject *args) { PyObject *input; PyArrayObject *array; double sum; int i, n; if (!PyArg_ParseTuple(args, "O", &input)) return NULL; array = (PyArrayObject *) PyArray_ContiguousFromObject(input, PyArray_DOUBLE, 2, 2); if (array == NULL) return NULL; n = array->dimensions[0]; if (n > array->dimensions[1]) n = array->dimensions[1]; sum = 0.; for (i = 0; i < n; i++) sum += *(double *)(array->data + i*array->strides[0] + i*array->strides[1]); Py_DECREF(array); return PyFloat_FromDouble(sum); } Last example: ============= static PyObject * matrix_vector(PyObject *self, PyObject *args) { PyObject *input1, *input2; PyArrayObject *matrix, *vector, *result; int dimensions[1]; double factor[1]; double real_zero[1] = {0.}; long int_one[1] = {1}; long dim0[1], dim1[1]; extern dgemv_(char *trans, long *m, long *n, double *alpha, double *a, long *lda, double *x, long *incx, double *beta, double *Y, long *incy); if (!PyArg_ParseTuple(args, "dOO", factor, &input1, &input2)) return NULL; matrix = (PyArrayObject *) PyArray_ContiguousFromObject(input1, PyArray_DOUBLE, 2, 2); if (matrix == NULL) return NULL; vector = (PyArrayObject *) PyArray_ContiguousFromObject(input2, PyArray_DOUBLE, 1, 1); if (vector == NULL) return NULL; if (matrix->dimensions[1] != vector->dimensions[0]) { PyErr_SetString(PyExc_ValueError, "array dimensions are not compatible"); return NULL; } dimensions[0] = matrix->dimensions[0]; result = (PyArrayObject *)PyArray_FromDims(1, dimensions, PyArray_DOUBLE); if (result == NULL) return NULL; dim0[0] = (long)matrix->dimensions[0]; dim1[0] = (long)matrix->dimensions[1]; dgemv_("T", dim1, dim0, factor, (double *)matrix->data, dim1, (double *)vector->data, int_one, real_zero, (double *)result->data, int_one); return PyArray_Return(result); } -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

3 2

floating point exception weirdness
by Andrew Straw Nov. 3, 2004

Nov. 3, 2004

Hi All, I've isolated numarray.ieeespecial as the cause of a strange error which I posted on c.l.py: http://groups.google.com/groups?dq=&hl=en&lr=&ie=UTF-8&threadm=41198B29.709… My non-expert guess as to an explanation: some statements early in numarray.ieeespecial (e.g. _na.array(1.0)/_na.array(0.0)) result in a floating point exception being set which lies dormant until I call certain functions in the Intel IPP library (e.g. ippiAddWeighted_8u32f_C1IR), at which point the latent exception bit my Python program is killed while the console says "Floating point exception". I've come to this tentative conclusion based on 3 minimal Pyrex programs: ---------------------------------------------------- Version 1: import numarray.numarrayall as _na plus_inf = inf = _na.array(1.0)/_na.array(0.0) cimport avg avg.work() # calls IPP, terminates with "Floating point exception". ---------------------------------------------------- Version 2: import numarray.numarrayall as _na # Define *ieee special values* _na.Error.pushMode(all="ignore") plus_inf = inf = _na.array(1.0)/_na.array(0.0) _na.Error.popMode() cimport avg avg.work() # calls IPP, terminates with "Floating point exception". ---------------------------------------------------- Version 3: import numarray.numarrayall as _na cimport avg avg.work() # calls IPP, terminates normally In the short term, I'll try and work around this problem by not importing numarray.ieeespecial, but in the long term, I think we should try and fix this. I'm happy to try to debug further if someone gives me advice on where to go from here. Cheers! Andrew

2 6

extracting a random subset of a vector
by Curzio Basso Sept. 8, 2004

Sept. 8, 2004

Hi all, I have an optimization problem. I currently use the following code to select a random subset of a rank-1 array: ---------------------------------------------------------- import numarray as NA import numarray.random_array as RA N = 1000 M = 100 full = NA.arange(N) subset = full[RA.permutation(N)][:M] --------------------------------------------------------- However, it's quite slow (at least with N~40k), and from the hotshot output is looks like it's the indexing, not the permutation, which takes time. Does anyone have a suggestion on a faster pure-python implementation? thanks

6 10

error bar default color
by Jin-chung Hsu Aug. 31, 2004

Aug. 31, 2004

In 0.61.0, when plotting a simple array with error bars, the default color of the error bars is black, instead of being the same as the line/markers color, e.g.: >>> errorbar([1,2,3,4,5],[3,4,5,6,7],fmt='ro',yerr=[1,1,1,1,1]) I prefer them to be the same, especially since the default color for marker/line is blue and a beginner may be surprised to see the different color. This may be related to my last posting regarding the marker edge color. JC

7 9

Help req: IDL --> Numeric/numarray
by RJ Aug. 31, 2004

Aug. 31, 2004

Hi all, The current IDL REBIN() thread reminded me to post this here... I'm trying to port Dr. Hongjie Xie's IDL/ENVI Implementation of the FFT Based Algorithm for Automatic Image Registration (it calculates shift, scale and rotation for 2d arrays) http://www.nmt.edu/%7Ehjxie/xie-paper.pdf ftp://ftp.iamg.org/VOL29/v29-08-10.zip to Python/numarray/Numeric and running into some subtle problems. For instance, the IDL ROT() requires some additional steps to emulate, and the MAX(array, position) as well. And, I'm not at all familiar with the correspondence of the FFT libraries... even the simple shift-only routine I translated http://rjs.org/astro/1004x/Python/register/shift_idl.py produces different (incorrect) x shift values than the IDL. My current code http://rjs.org/astro/1004x/Python/register/ conforms to Xie's flow as much as possible for now, but should and will be re-factored when functional; his LogPolar function, to convert rectangular coordinate arrays to polar, is particularly slow. I was recently informed that IDL has a usable demo version which I will try this week, but I would love to hear if anyone else has interest in this. I think that the algorithm would make a nice standalone module, or perhaps something for nd_image. Xie's main examples are registration of satellite Earth imagery, but it should work just as well looking up ;-) Thank you, Ray Secret anti-spam filter-passing text. Include with reply: qwertyuiop

1 0

Help req: IDL --> Numeric/numarray
by RJ Aug. 31, 2004

Aug. 31, 2004

Hi all, The current IDL REBIN() thread reminded me to post this here... I'm trying to port Hongjie Xie's IDL/ENVI Implementation of the FFT Based Algorithm for Automatic Image Registration http://www.nmt.edu/%7Ehjxie/xie-paper.pdf ftp://ftp.iamg.org/VOL29/v29-08-10.zip to Python/numarray/Numeric and running into some subtle problems. For instance, the IDL ROT() requires some additional steps to emulate, and the MAX(array, position) as well. And, I'm not at all familiar with the correspondence of the FFT libraries... even the simple shift-only routine http://rjs.org/astro/1004x/Python/register/shift_idl.py produces different (incorrect) x shift values than the IDL. My current code http://rjs.org/astro/1004x/Python/register/ conforms to Xie's flow as much as possible for now, but should and will be re-factored when functional. His LogPolar function, to convert rectangular coordinate arrays to polar, is particularly slow. I was recently informed that IDL has a usable demo version which I will try this week, but I would love to hear if anyone else has interest in this. I think that the algorithm would make a nice standalone module, or perhaps something for nd_image, as it calculates shift, scale and rotation for 2d arrays. Xie's main examples are registration of satellite Earth imagery, but it should work just as well looking up ;-) Thank you, Ray Secret anti-spam filter-passing text. Include with reply: qwertyuiop

1 0

RE: [Numpy-discussion] rebin
by Nadav Horesh Aug. 28, 2004

Aug. 28, 2004

For the most general form of binning I use a convolution (by a 2D mask) followed by a subsmapling. For example for a 3x3 binning: mask = ones((3,3)) binned = convolve2d(data,mask,'same')[1::3,1::3] Nadav. -----Original Message----- From: Russell E. Owen [mailto:owen@astro.washington.edu] Sent: Sat 28-Aug-04 03:34 To: numpy-discussion(a)lists.sourceforge.net Cc: Subject: [Numpy-discussion] rebin Any suggestions on an efficient means to bin a 2-d array? REBIN is the IDL function I'm trying to mimic. Binning allows one to combine sets of pixels from one array to form a new array that is smaller by a given factor along each dimension. To nxm bin a 2-dimensional array, one averages (or sums or ?) each nxm block of entries from the input image to form the corresponding entry of the output image. For example, to 2x2 bin a two-dimensional image, one would: average (0,0), (0,1), (1,0), (1,1) to form (0,0) average (0,2), (0,3), (1,2), (1,3) to form (0,1) ... In case it helps, in my immediate case I'm binning a boolean array (a mask) and thus can live with almost any kind of combination. -- Russell ------------------------------------------------------- This SF.Net email is sponsored by BEA Weblogic Workshop FREE Java Enterprise J2EE developer tools! Get your free copy of BEA WebLogic Workshop 8.1 today. http://ads.osdn.com/?ad_id=5047&alloc_id=10808&op=click _______________________________________________ Numpy-discussion mailing list Numpy-discussion(a)lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion

1 0

marker edge color
by Jin-chung Hsu Aug. 27, 2004

Aug. 27, 2004

In 0.61.0, the marker edge color is still black, instead of being the same as the marker face color. Personally, I think they should be the same. What do people think? One reason for the same color argument is that if the markersize is set to a small value, it will show mostly the edge color. One possible reason against it is that if the marker color is white (or whatever the background color is), then you can't see the marker. But we can default this case to black (or whatever). JC Hsu

1 0

Confusion regarding Numeric array resizing
by Jp Calderone Aug. 26, 2004

Aug. 26, 2004

I'm confused by the restriction on resizing arrays demonstrated below: >>> from Numeric import array >>> a = array([0]) >>> a.resize((2,)) >>> b = a >>> a.resize((3,)) Traceback (most recent call last): File "<stdin>", line 1, in ? ValueError: cannot resize an array that has been referenced or is referencing another array in this way. Use the resize function. >>> del b >>> a.resize((3,)) >>> a array([0, 0, 0]) This seems like an unnecessary restriction. Is there a reason I'm missing for it to be in place? Jp

2 1

RE: [Numpy-discussion] fromstring doesn't accept buffer object
by Nadav Horesh Aug. 26, 2004

Aug. 26, 2004

Sorry for mixing test and production systems: There is no problem here with python 2.3, I got this failure with python 2.4a2: ========================================== import numarray as n import Numeric as N s = '12341234' b = buffer(s) print 'numarray: fromstring (string ..) :' print n.fromstring(s, type=n.Float64) print 'Numeric: fromstring (string ..) :' print N.fromstring(s, typecode=N.Float64) print 'Numeric: fromstring (buffer ..) :' print N.fromstring(b, typecode=N.Float64) print 'numarray: fromstring (buffer ..) :' print n.fromstring(b, type=n.Float64) === Results: ==== numarray: fromstring (string ..) : [ 3.05810932e-57] Numeric: fromstring (string ..) : [ 3.05810932e-57] Numeric: fromstring (buffer ..) : [ 3.05810932e-57] === This error happens only with Python 2.4.a2: numarray: fromstring (buffer ..) : Traceback (most recent call last): File "test.py", line 17, in ? print n.fromstring(b, type=n.Float64) File "/usr/local/lib/python2.4/site-packages/numarray/numarraycore.py", line 378, in fromstring arr._data, 0, (type.bytes,), type.bytes) libnumarray.error: copyNbytes: access beyond buffer. offset=7 buffersize=0 Nadav. -----Original Message----- From: Todd Miller [mailto:jmiller@stsci.edu] Sent: Thu 26-Aug-04 15:51 To: Nadav Horesh Cc: numpy-discussion Subject: Re: [Numpy-discussion] fromstring doesn't accept buffer object This is what I see this morning using CVS: >>> from numarray import * >>> s = "thisthis" >>> fromstring(buffer(s), shape=(2,), type=Int32) array([1936287860, 1936287860]) What kind of failure are you seeing? Regards, Todd On Thu, 2004-08-26 at 09:28, Nadav Horesh wrote: > Numerc accepts a buffer object instead of a string: > > arr = Numeric.fromstring(buffer(.some_string), ..) > while numarray doesn't. > > Is it intentional? > > Nadav. > > > ------------------------------------------------------- > SF.Net email is sponsored by Shop4tech.com-Lowest price on Blank Media > 100pk Sonic DVD-R 4x for only $29 -100pk Sonic DVD+R for only $33 > Save 50% off Retail on Ink & Toner - Free Shipping and Free Gift. > http://www.shop4tech.com/z/Inkjet_Cartridges/9_108_r285 > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion --

1 0