Mailman 3 December 2004 - NumPy-Discussion

PyArray_ContiguousFromObject & PyDECREF
by Michiel Jan Laurens de Hoon 16 May '11

16 May '11

Dear pythoneers, In Chapter 12 ("Writing a C extension") in the Numerical Python manual, in the first example PyArray_ContiguousFromObject is used together with PyDECREF, while in the last example PyArray_ContiguousFromObject is used but the arrays are not PyDECREF'ed before the function returns. From Chapter 13 ("C API Reference") it seems that a PyDECREF is needed in both cases. Or am I missing something here? Is PyDECREF not needed for some reason in the last example? Thanks in advance, Michiel de Hoon U Tokyo First example: ============== static PyObject * trace(PyObject *self, PyObject *args) { PyObject *input; PyArrayObject *array; double sum; int i, n; if (!PyArg_ParseTuple(args, "O", &input)) return NULL; array = (PyArrayObject *) PyArray_ContiguousFromObject(input, PyArray_DOUBLE, 2, 2); if (array == NULL) return NULL; n = array->dimensions[0]; if (n > array->dimensions[1]) n = array->dimensions[1]; sum = 0.; for (i = 0; i < n; i++) sum += *(double *)(array->data + i*array->strides[0] + i*array->strides[1]); Py_DECREF(array); return PyFloat_FromDouble(sum); } Last example: ============= static PyObject * matrix_vector(PyObject *self, PyObject *args) { PyObject *input1, *input2; PyArrayObject *matrix, *vector, *result; int dimensions[1]; double factor[1]; double real_zero[1] = {0.}; long int_one[1] = {1}; long dim0[1], dim1[1]; extern dgemv_(char *trans, long *m, long *n, double *alpha, double *a, long *lda, double *x, long *incx, double *beta, double *Y, long *incy); if (!PyArg_ParseTuple(args, "dOO", factor, &input1, &input2)) return NULL; matrix = (PyArrayObject *) PyArray_ContiguousFromObject(input1, PyArray_DOUBLE, 2, 2); if (matrix == NULL) return NULL; vector = (PyArrayObject *) PyArray_ContiguousFromObject(input2, PyArray_DOUBLE, 1, 1); if (vector == NULL) return NULL; if (matrix->dimensions[1] != vector->dimensions[0]) { PyErr_SetString(PyExc_ValueError, "array dimensions are not compatible"); return NULL; } dimensions[0] = matrix->dimensions[0]; result = (PyArrayObject *)PyArray_FromDims(1, dimensions, PyArray_DOUBLE); if (result == NULL) return NULL; dim0[0] = (long)matrix->dimensions[0]; dim1[0] = (long)matrix->dimensions[1]; dgemv_("T", dim1, dim0, factor, (double *)matrix->data, dim1, (double *)vector->data, int_one, real_zero, (double *)result->data, int_one); return PyArray_Return(result); } -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

3 2

ANN: matplotlib-0.70
by John Hunter 30 Dec '04

30 Dec '04

matplotlib is a 2D graphics package that produces plots from python scripts, the python shell, or embeds them in your favorite python GUI -- wx, gtk, tk, fltk currently supported with qt in the works. Unlike many python plotting alternatives is written in python, so it is easy to extend. matplotlib is used in the finance industry, web application servers, and many scientific and enginneering disciplines. With a large community of users and developers, matplotlib is approaching the goal of having a full featured, high quality, 2D plotting library for python. A lot of development has gone into matplotlib since the last major release, which I'll summarize here. For details, see the notes for the incremental releases at http://matplotlib.sf.net/whats_new.html. Major changes since matplotlib-0.60 - The alpha version of the users guide - http://matplotlib.sf.net/users_guide.pdf. There are still a number of sections to be completed, but it's a start! - The matlab namespace renamed pylab - if you are upgrading from a version older than 0.64, please remove site-packages/matplotlib before upgrading. See http://matplotlib.sourceforge.net/matlab_to_pylab.py - New plot types: contour plots (contour), polar charts (polar), horizontal bar charts (barh), pie charts (pie), sparse matrix visualization (spy and spy2). Eg, http://matplotlib.sf.net/screenshots.html#polar_demo - Full ipython http://ipython.scipy.org integration in the "pylab" mode for interactive control of matplotlib plots from the python shell. - A significantly improved interactive toolbar for panning, zooming, zoom to rect - see http://matplotlib.sf.net/tutorial.html#toolbar2. - New backends: FLTK, Cairo, GTKCairo - Text - rotated mathtext, mathtext for postscript, text bounding boxes - Colormaps - 14 colormaps built-in http://matplotlib.sf.net/screenshots.html#pcolor_demo - Images - performance optimizations for 4x faster large image handling, PIL support, interpolation and colorbar improvements, imread - Event handling for capturing mouse clicks, movements, keypresses, etc. - same pylab interface works across GUIs. See examples/keypress_demo.py, examples/picker_demo.py, examples/coords_demo.py - set and get matlab style property introspection - http://matplotlib.sf.net/examples/set_and_get.py - improved dates handling for dates and date string formatting from 0000-9999, eg http://matplotlib.sf.net/screenshots.html#finance_work - Be sure to check out the 120 examples at http://matplotlib.sf.net/examples Home page : http://matplotlib.sourceforge.net Downloads : http://sourceforge.net/projects/matplotlib Screenshots : http://matplotlib.sourceforge.net/screenshots.html Tutorial : http://matplotlib.sourceforge.net/tutorial.html Credits : http://matplotlib.sourceforge.net/credits.html John Hunter

1 0

log bug?
by Ralf Juengling 26 Dec '04

26 Dec '04

Hi, I'm not sure this behavior is intended: >>> from numarray import * >>> log(array([0.0], type=Float64)) Warning: Encountered divide by zero(s) in log array([ inf]) >>> -inf would be a more useful result. ralf

2 1

Re: [Numpy-discussion] Striding on NumArray objects
by Todd Miller 21 Dec '04

21 Dec '04

> > This is a difficult question for me, but I think the answer is that the > > use of _bytestride is very limited. _bytestride is used to compute the > > "natural" strides of an almost contiguous array, e.g. a field of a > > recarray. That is, given a bytestride and a shape, the strides of a > > field of a contiguous RecArray are implied. > > Yes, I was trying to use _bytestride mainly in RecArray contexts. It just > happens that I've used a NumArray object to check the _bytestride behaviour. > > > However, once we start slicing (say in more than one dimension), > > _strides contains more and more information and is no longer implied by > > just the shape and bytestride but also by the history of slicing. From > > that perspective, it's not clear what _bytestride can be relied upon > > for in general or that it needs to be (or can be) kept up to date during > > slicing. > > Well, for the time being it seems that RecArray does not support > multidimensionalty, so I can just use _bytestride as a shortcut of > _strides[0] here. I wouldn't. I had a veritable stroke of genius and talked to JC Hsu... He suggested doing concrete examples and also indicated that full multi-dimensional RecArray support is almost all (except for repr'ing) there. To be clear, in general, _bytestride != _itemsize. For a RecArray field (of a contiguous RecArray) where each row contains just one element, generally _strides[-1] == _bytestride != _itemsize: >>> r = rec.array(buffer=None, formats="i2,(3,4)f4,a3", shape=10) >>> r.field(0).info() class: <class 'numarray.numarraycore.NumArray'> shape: (10,) strides: (53,) byteoffset: 0 bytestride: 53 itemsize: 2 aligned: 0 contiguous: 0 data: <memory at 0x08256c10 with size:0x00000212 held by object 0x40764db8 aliasing object 0x00000000> byteorder: little byteswap: 0 type: Int16 For a RecArray field where each row contains more than one element, _strides[-1] == itemsize: >>> r.field(1).info() class: <class 'numarray.numarraycore.NumArray'> shape: (10, 3, 4) strides: (53, 16, 4) byteoffset: 2 bytestride: 53 itemsize: 4 aligned: 0 contiguous: 0 data: <memory at 0x08256978 with size:0x00000212 held by object 0x407613a8 aliasing object 0x00000000> byteorder: little byteswap: 0 type: Float32 The situation gets more complicated still when someone starts slicing the field or RecArray: >>> x[::3].info() class: <class 'numarray.numarraycore.NumArray'> shape: (4, 3, 4) strides: (159, 16, 4) byteoffset: 2 bytestride: 53 itemsize: 4 aligned: 0 contiguous: 0 data: <memory at 0x08256c10 with size:0x00000212 held by object 0x40764db8 aliasing object 0x00000000> byteorder: little byteswap: 0 type: Float32 Multi-dimensional RecArrays add their own twist: >>> r = rec.array(buffer=None, formats="i2,(3,4)f4,a3",shape=(10,10,10)) >>> r.field(1).info() class: <class 'numarray.numarraycore.NumArray'> shape: (10, 10, 10, 3, 4) strides: (5300, 480, 48, 16, 4) byteoffset: 2 bytestride: 53 itemsize: 4 aligned: 0 contiguous: 0 data: <memory at 0x08257dd0 with size:0x0000cf08 held by object 0x40763a70 aliasing object 0x00000000> byteorder: little byteswap: 0 type: Float32 HTH, Todd

2 1

Striding on NumArray objects
by Francesc Altet 20 Dec '04

20 Dec '04

Hi, I'm a bit lost with the next example: In [28]: from numarray import * In [29]: a=arange(10) In [30]: a.iscontiguous() Out[30]: 1 In [31]: b=a[::2] In [32]: b.iscontiguous() Out[32]: 0 That seems to suggest that b shares the same data buffer than a. Indeed: In [36]: a._data Out[36]: <memory at 0x082494d8 with size:0x00000028 held by object 0xb762c260 aliasing object 0x00000000> In [37]: b._data Out[37]: <memory at 0x082494d8 with size:0x00000028 held by object 0xb762c260 aliasing object 0x00000000> At this point, I believe that _bytestride should be different on both arrays, but: In [33]: a._bytestride Out[33]: 4 In [34]: b._bytestride Out[34]: 4 while I expected to find b._bytestride equal to 8. Is that an error or a lack of understanding on my part? Cheers, -- Francesc Altet >qo< http://www.carabos.com/ Cárabos Coop. V. V V Enjoy Data ""

3 3

Re: [Numpy-discussion] ufuncs on user-defined types in numarray
by Gary Ruben 20 Dec '04

20 Dec '04

First, thanks Todd for elevating this to 'requested feature' status. Konrad, I had no idea what interval arithmetic is, so I looked it up. I'm not sure if what I've done is considered interval arithmetic - I suspect it is, but that I've only implemented it partially. The code isn't that clever. It doesn't care about propagating rounding errors correctly at the error limits. It just implements a brute-force approach to calculating operations on the error limits of a data-point value along with the central value. This is already more accurate than what most people do, which is to linearise the error limits and treat them as infinitessimals. The central and outer values are grouped in an object and any operations are applied, without any consideration of rounding errors, to all three values for every operation. Gary ----- Original Message ----- > > I didn't realise this was possible. I should use it on my > > ErrorVal module <http://users.bigpond.net.au/gazzar/python.html>. > > Last time I tried (quite a > > Indeed! You could then even have arrays of ErrorVal objects with no > extra effort, at least as long as ErrorVal doesn't implement the > sequence protocol (because then all the Numeric functions will > silently convert them to arrays). > > Your module looks interesting. What does it do internally, interval > arithmetic? > > Konrad. -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm

1 0

Re: [Numpy-discussion] ufuncs on user-defined types in numarray
by Gary Ruben 20 Dec '04

20 Dec '04

Thanks Konrad, I didn't realise this was possible. I should use it on my ErrorVal module <http://users.bigpond.net.au/gazzar/python.html>. Last time I tried (quite a while ago), numarray's object array support wasn't up to allowing me to port ErrorVal from Numeric. I don't think it matters since I don't know of anyone, besides me, who has tried using my module :-( but I thought I'd mention it for Todd's benefit since I'm one who could benefit from implementing it in numarray. Gary R. ----- Original Message ----- From: "Todd Miller" <jmiller(a)stsci.edu> To: konrad.hinsen(a)laposte.net Subject: Re: [Numpy-discussion] ufuncs on user-defined types in numarray Date: 17 Dec 2004 10:09:51 -0500 > > On Fri, 2004-12-17 at 08:31, konrad.hinsen(a)laposte.net wrote: > > I am working on making ScientificPython > > (http://dirac.cnrs-orleans.fr/ScientificPython/) compatible with > > numarray. One problem I stumbled across is that one feature of > > Numeric seems to be absent from numarray: the possibility to call > > ufuncs with arguments of non-numeric types, leading to a > > corresponding method call. > > > > As an illustration, the following example works fine with > > Numeric, but crashes with numarray: > > > > from Numeric import sqrt > > #from numarray import sqrt > > > > class DummyValue: > > > > def __init__(self, string): > > self.string = string > > > > def __str__(self): > > return self.string > > > > def sqrt(self): > > return DummyValue('sqrt(%s)' % self.string) > > > > x = DummyValue('x') > > print sqrt(x) > > > > > > Is this a bug or a feature? > > It's a missing feature. > > > Is there another way to make ufuncs work with user-defined classes and types? > > Not that I know of. How critical a feature is it? Do you have a work around? > > numarray ufunc handling is in flux as I'm adding numarray support to > scipy. At some point, there's going to be a major ufunc consolidation, > and perhaps I can add this additional feature then. > > Regards, > Todd > > > > ------------------------------------------------------- > SF email is sponsored by - The IT Product Guide > Read honest & candid reviews on hundreds of IT Products from real users. > Discover which products truly live up to the hype. Start reading now. > http://productguide.itmanagersjournal.com/ > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion -- ___________________________________________________________ Sign-up for Ads Free at Mail.com http://promo.mail.com/adsfreejump.htm

3 2

Numeric 23.6 for Python 2.4
by Jonathan Brandmeyer 20 Dec '04

20 Dec '04

Does the Numeric team intend to provide a build of Numeric 23.6 for Python 2.4? If not, I have managed to build a release using the free-as-in-beer version of MSVC 7.1, aka MS Visual C++ Toolkit 2003. I basically followed the directions here: http://www.vrplumber.com/programming/mstoolkit/index.html and altered setup.py to use Numeric's own math libraries rather than the atlas libs. The build has been tested only by running the supplied Test/test.py program after installing the package. For the moment, you can download the file from http://www4.ncsu.edu/~jdbrandm/Numeric-23.6.win32-py2.4.exe but I would prefer it to be hosted on the Sourceforge download page for the project. Its md5sum is 2df692d7b8136b68e3b511ed27cbbdd8 Thanks, -Jonathan Brandmeyer P.S. I apologize if the message's formatting is borked up; I'm sending this from a webmail interface.

1 0

Re: simple numarray bug
by Todd Miller 17 Dec '04

17 Dec '04

On Fri, 2004-12-17 at 18:28 -0800, Andrew Straw wrote: > Hi Todd, > Hi Andrew, > (Nice meeting you at SciPy '04!) Yes, nice meeting you. > I think this is a simple bug which I just filed a bug report on SF for: > http://sourceforge.net/tracker/index.php?func=detail&aid=1087378&group_id=1… > > This email is just a "heads up" that I created the bug report. Thanks for the bug report and heads up. Source Forge automatically gives me e-mail for numarray bug reports so "heads ups" are not necessary. However, I often fall behind real time response so if you make a report and nothing happens for a while gentle nagging is OK. I committed a "fix" for this to CVS... There is now a LinearAlgebraError subclass of Exception aliased as LinAlgError. Regards, Todd

1 0

copying ctypes arrays to numarray?
by RJ 17 Dec '04

17 Dec '04

I'm posting here to see if numarray has a method that could work like array.buffer_info(). numarray.info() returns a text output that can't be used like the array method is to memmove() between ctypes and Python arrays without parsing, apparently. ctypes thread with Thomas Heller below. His main question: "Maybe you should ask to make sure that there's no way to copy between objects implementing the buffer protocol with some Python function that I do not know about?"

4 6