Mailman 3 October 2004 - NumPy-Discussion

PyArray_ContiguousFromObject & PyDECREF
by Michiel Jan Laurens de Hoon 16 May '11

16 May '11

Dear pythoneers, In Chapter 12 ("Writing a C extension") in the Numerical Python manual, in the first example PyArray_ContiguousFromObject is used together with PyDECREF, while in the last example PyArray_ContiguousFromObject is used but the arrays are not PyDECREF'ed before the function returns. From Chapter 13 ("C API Reference") it seems that a PyDECREF is needed in both cases. Or am I missing something here? Is PyDECREF not needed for some reason in the last example? Thanks in advance, Michiel de Hoon U Tokyo First example: ============== static PyObject * trace(PyObject *self, PyObject *args) { PyObject *input; PyArrayObject *array; double sum; int i, n; if (!PyArg_ParseTuple(args, "O", &input)) return NULL; array = (PyArrayObject *) PyArray_ContiguousFromObject(input, PyArray_DOUBLE, 2, 2); if (array == NULL) return NULL; n = array->dimensions[0]; if (n > array->dimensions[1]) n = array->dimensions[1]; sum = 0.; for (i = 0; i < n; i++) sum += *(double *)(array->data + i*array->strides[0] + i*array->strides[1]); Py_DECREF(array); return PyFloat_FromDouble(sum); } Last example: ============= static PyObject * matrix_vector(PyObject *self, PyObject *args) { PyObject *input1, *input2; PyArrayObject *matrix, *vector, *result; int dimensions[1]; double factor[1]; double real_zero[1] = {0.}; long int_one[1] = {1}; long dim0[1], dim1[1]; extern dgemv_(char *trans, long *m, long *n, double *alpha, double *a, long *lda, double *x, long *incx, double *beta, double *Y, long *incy); if (!PyArg_ParseTuple(args, "dOO", factor, &input1, &input2)) return NULL; matrix = (PyArrayObject *) PyArray_ContiguousFromObject(input1, PyArray_DOUBLE, 2, 2); if (matrix == NULL) return NULL; vector = (PyArrayObject *) PyArray_ContiguousFromObject(input2, PyArray_DOUBLE, 1, 1); if (vector == NULL) return NULL; if (matrix->dimensions[1] != vector->dimensions[0]) { PyErr_SetString(PyExc_ValueError, "array dimensions are not compatible"); return NULL; } dimensions[0] = matrix->dimensions[0]; result = (PyArrayObject *)PyArray_FromDims(1, dimensions, PyArray_DOUBLE); if (result == NULL) return NULL; dim0[0] = (long)matrix->dimensions[0]; dim1[0] = (long)matrix->dimensions[1]; dgemv_("T", dim1, dim0, factor, (double *)matrix->data, dim1, (double *)vector->data, int_one, real_zero, (double *)result->data, int_one); return PyArray_Return(result); } -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

3 2

floating point exception weirdness
by Andrew Straw 02 Nov '04

02 Nov '04

Hi All, I've isolated numarray.ieeespecial as the cause of a strange error which I posted on c.l.py: http://groups.google.com/groups?dq=&hl=en&lr=&ie=UTF-8&threadm=41198B29.709… My non-expert guess as to an explanation: some statements early in numarray.ieeespecial (e.g. _na.array(1.0)/_na.array(0.0)) result in a floating point exception being set which lies dormant until I call certain functions in the Intel IPP library (e.g. ippiAddWeighted_8u32f_C1IR), at which point the latent exception bit my Python program is killed while the console says "Floating point exception". I've come to this tentative conclusion based on 3 minimal Pyrex programs: ---------------------------------------------------- Version 1: import numarray.numarrayall as _na plus_inf = inf = _na.array(1.0)/_na.array(0.0) cimport avg avg.work() # calls IPP, terminates with "Floating point exception". ---------------------------------------------------- Version 2: import numarray.numarrayall as _na # Define *ieee special values* _na.Error.pushMode(all="ignore") plus_inf = inf = _na.array(1.0)/_na.array(0.0) _na.Error.popMode() cimport avg avg.work() # calls IPP, terminates with "Floating point exception". ---------------------------------------------------- Version 3: import numarray.numarrayall as _na cimport avg avg.work() # calls IPP, terminates normally In the short term, I'll try and work around this problem by not importing numarray.ieeespecial, but in the long term, I think we should try and fix this. I'm happy to try to debug further if someone gives me advice on where to go from here. Cheers! Andrew

2 6

Re: [Numpy-discussion] Counting array elements
by Todd Miller 02 Nov '04

02 Nov '04

I have returned from our astronomical data systems conference and I am going to take a short cut and summarize what I saw as the key developments of this thread. I apologize for not responding sooner and individually but the web-mail system I use isn't effective for conducting any kind of discussion. You guys did a great job sorting this out this week. I marked my key points with **. The rest is probably only for people with a lot of patience. ** I've finally come to terms with the fact that functions are the right way to do numarray rather than methods. The arguments in the Numeric manual are no more persuasive now than they ever were, but Stephen Walton's remarks about method explosion finally convinced me what the "real" reason for doing functions is that using methods combines every new feature under the umbrella of a single namespace, the NumArray class. Using functions lets us partition things into modules which can be used selectively and makes a more extensible and understandable system. Thanks Stephen. A couple people remarked that using .flat might solve everything with something like a.flat.sum() or sum(ravel(a). This gets to the original motivation for the sum() method, which was the codification of a simple and storage efficient technique for reducing noncontiguous arrays. The first point is that a non-contiguous array cannot generally be reshaped without making a copy. The basic idea of the sum() method is to do *two* reductions, the first, along a single axis, results in a smaller contiguous array. In the case of astronomical images which are generally square or at least non-degenerate, the reduction result is a *much* smaller array. The second reduction handles all the remaining dimensions since .flat is guaranteed to work because the array is contiguous. The end result is a complete sum() without righting additional ufuncs or making an array copy. There was understandable confusion about why .flat is sometimes allowed to fail. Since it is an attribute, we thought it inappropriate to make it return a copy of the source array and chose instead to raise an exception. In contrast, it is reasonable for the ravel() function to return a completely different array, so it always works. (I just noticed that ravel() is not named flat()). Some of our more contemporary thinkers suggested using iterators to produce a .flat which always works. If anyone has an idea how to make this work with good performance, please let me know; I don't. ** Tim Hochberg pointed out that we can overload the reduction (and not accumulation?) axis parameter with an "all" or a tuple describing a sequence of axes to reduce along. My perception was that there was a consensus behind this and in any case I'm in agreement with Tim. Alan Isaac pointed out that None might be better here than "all" and I agree. At this point, I think sumAll() is dead, the sum() method will be deprecated, and the reductions should be expanded as Tim suggested. ** Peter Verveer made some comments about the expectations of a naive user regarding reductions, namely that "all" should be the default. My own experience bears this out, and I am torn about what to do here. Chris Barker pointed out the need for backward compatibility with Numeric, and given the current numarray goal of supporting SciPy, this need is growing stronger and more complex. SciPy uses yet another axis convention. If anyone has any ideas how to handle these multiple conventions with elegance, let me know. A number of people commented on our naming conventions, an issue which we have side stepped for the moment with sumAll(). My impression is that, for better or worse, numarray uses the lowerUpper() version of Camel case. I think this is very much a matter of personal taste and don't claim to have any. My guess is that numarray is probably inconsistent at the moment, in part because lowerUpper() often degenerates into merely lower() which degenerates into confusion. Regards, Todd

5 7

Re: floating point exception weirdness (Andrew Straw)
by Steve Chaplin 01 Nov '04

01 Nov '04

> Just a small addendum, (which I hope will spur on bug-fixing once Todd > et al. are back from the conference -- let me know if I should file a > sourceforge bug report): I've not read all this thread so I don't know the full background. But I had a floating point / SSE problem using numarray. It turned out to be a glibc not numarray problem and was solved by upgrading glibc. http://sources.redhat.com/bugzilla/show_bug.cgi?id=10 There was also a SourceForge bug report but I can't locate it. Regards Steve

2 2

Re: [Numpy-discussion] Counting array elements
by Tim Hochberg 31 Oct '04

31 Oct '04

Robert Kern wrote: > Tim Hochberg wrote: > >> Could you describe the SciPy axis convention: I'm not familiar with it. > > > axis=-1 OK, so Numarray (currently) and Numeric use axis=0, SciPy uses axis=-1 and there is some desire to use axis=ALL as instead. One advantage of ALL is that it breaks everyone's code equally, so there wouldn't be any charges of favoritism <0.8 wink>. I can't come up with any way to reconcile the three, but I can suggest a transition strategy whatever the decision. Supply an option so that one can require axis arguments to all calls to reduce. Then it's relatively easy to track down all the reduce calls and fix the ones that are broken. Something like numarray.setRequireReduceAxisArg(True). FWIW, it wouldn't bother me much to use SciPy's default here: supporting SciPy is a worthwhile goal and I think SciPy's choice here is a reasonable one. Another alternative that wouldn't bother me much is "In the face of ambiguity, refuse the temptation to guess". That is, always require axis arguments for multidimensional arrays. While not backwards compatible, this would make the transition relatively easy, since uses that might fail would raise exceptions. -tim

2 1

vector cross product
by Gary Ruben 30 Oct '04

30 Oct '04

Not that I have a really urgent need, but is there a reason that nice, fast C-based vector operations aren't implemented in Numeric or numarray? I notice Fernando Perez has a cross product as a useful SciPy weave example on his site. I've also seen comments elsewhere about Numpy's lack of a cross product. eg. <http://mail.python.org/pipermail/python-list/2004-March/213878.html> I'm using Konrad Hinsen's Scientific Python for the convenience value of his Vector class, which also provides a nice angle() method but it bothers me that it's implemented in native Python. The Vector type in vpython probably does it 'properly', but I don't use it just for the convenience since it adds an extra dependency to my code. comments? Gary R. -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm

1 0

bug? in len(arr.flat)
by Sebastian Haase 29 Oct '04

29 Oct '04

Hi, I have a (UInt16) 3d data stack and want to get to it's underlying buffer (to (later) feed it into memmap) ... I noticed that len(pr2._flat) is half of len(pr2._data) - like it doesn't multiply itemsize in. >>> pr2.shape (40, 512, 512) >>> pr2.flat.shape (10485760) >>> 512*512*40 10485760 >>> len(pr2.flat) 10485760 >>> pr2.flat._itemsize 2 >>> len(pr2._data) 20971520 >>> pr2._byteoffset 0 Is this a bug or am I missunderstanding ? Thanks, Sebastian Haase

2 2

Re: [Numpy-discussion] Counting array elements
by Chris Barker 26 Oct '04

26 Oct '04

Stephen Walton wrote: > There is a difference between the sum() Ufunc and the sum() method which > is not mentioned in the documentation: the function works along an > axis, while the method works on the whole array. That is, A.sum() and > A.flat.sum() are equivalent regardless of the rank of A. Bummer. I was hoping this was a move to a more object-oriented style, rather than different functionality. Also, it's pretty confusing terminology, particularly if it's not documented! Why not .SumAll() or something? -Chris -- Christopher Barker, Ph.D. Oceanographer NOAA/OR&R/HAZMAT (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

10 31

Singaporeans ay tumutulong para mapaunlad ang sariling negosyo
by Mei Kuan 25 Oct '04

25 Oct '04

Dear Filipino friend, Kumusta ka na? We were looking and your email just appeared, perhaps it was GOD's will. We sincerely hope that you read on this letter. This may be of significant relevance to you and your loved ones and give you something you are looking for in life. Do allow us to provide you with a brief introduction of ourselves. We are a team of Singaporean entrepreneurs hailing from various professional fields. We know that, in the new millennium, more Filipino employees and professionals are finding it harder to get ahead in life due to greater job insecurity as a result of corporate downsizing and global outsourcing, diminishing wages, office politics, not forgetting constant retrenchment threats. They are further affected by the rising costs of living and interest rates, not forgetting the current economic difficulties that Philippines is currently facing. There are also thousands of Filipinos who have to endure the heart-break of leaving their loved ones to venture overseas in order to support their loved ones and the Philippines economy once again. Filipino businessmen too, have to grapple with increasing economic and political uncertainties, epidemic threats such as the Avian Flu, competitive threats and unstable crude oil crisis. Further, due to the increasingly rapid changes in the business environment, they find it harder to keep up with the increasingly volatile business cycles. We recognise these problems faced by many Filipinos today and decide to embark on a more fulfilling long term career of helping them solve their problems and improving their lives in the process. What we do is to help Filipinos develop/diversify into their own businesses in a new, potentially huge and expanding industry so that they can start managing the above adversities and making significant progress towards what they and their loved ones want in life once again. Would this be something that may be deemed as a long term solution in your life? Our fellow associates from Singapore will be flying specially to the Philippines to conduct a series of exclusive previews in Makati, Cebu and Naga in November. Would you be interested to attend one of our previews for you to discover how our revolutionary platform can possibly help you and your loved ones improve your results on a long-term basis? If you are interested to attend, could you kindly provide us with your cellphone no. for our senior associate, Mr. Chew to text you when he is in Philippines next month? Mr. Chew was a very successful corporate executive from a Multi-National Corporation and a former Economic Lecturer. He held a Master of Science Degree in Financial Economics. Hence, he knows what it takes for a business to be considered a viable one and of course, what it takes to succeed in the business. He gave up a very successful corporate life to help many Filipinos change their lives. Despite his busy schedule, he is committed to flying to Philippines to help them. As such, he is a great mentor, inspirational, dynamic leader to many of us. He gained great respects from many of our Filipinos and non Filipinos friends. We believe he is the best person to share with you in depth how our revolutionary platform can fulfill your goals in life and turn your dreams into reality. We would handle all enquiries via Chikka: 001877961 or Skype: Reychell We sincerely urge you to communicate with us on Chikka/Skype to know you better as a friend and understand the challenges you are currently facing because we are looking to help you on a long-term basis. Ingats. GOD BLESS. Chow Mei Kuan (Ms) / Don (Mr.) Email: reychell(a)singnet.com.sg /chewlw(a)singnet.com.sg Chikka No.: 001877961 Skype ID: Reychell P.S.: This may be a GOD-send opportunity to improve your life. Disclaimer: This email, together with any attachments, is intended ONLY for the use of the individual or entity to which it is addressed, and may contain information that is legally privileged, confidential, and/or subject to copyright. If you are not the intended recipient, please be informed that any dissemination, distribution or copying of this email, any attachment, or part thereof is strictly prohibited. Kindly note that internet communications are not secure, and therefore are susceptible to alterations. If you have received this email in error, please advise the sender by reply email, and delete this message. Your co-operation on this matter is highly appreciated. Thank you.

1 0

RE: [Numpy-discussion] Problems with complex matrices
by Sebastien.deMentendeHorne＠electrabel.com 21 Oct '04

21 Oct '04

gmres returns a tuple so you should have used res = dot(A, x[0]) - r seb > -----Original Message----- > From: Nils Wagner [mailto:nwagner@mecha.uni-stuttgart.de] > Sent: vendredi 22 octobre 2004 11:13 > To: SciPy Users List; numpy-discussion(a)lists.sourceforge.net > Subject: [Numpy-discussion] Problems with complex matrices > > > Hi all, > > Another bug is revealed > > Traceback (most recent call last): > File "complex_it.py", line 6, in ? > res=dot(A,x)-r > File > "/usr/lib/python2.3/site-packages/Numeric/dotblas/__init__.py", > line 55, in dot > if multiarray.array(a).shape == () or > multiarray.array(b).shape == (): > TypeError: a float is required > > Nils > >

1 0