Mailman 3 July 2008 - NumPy-Discussion

Newly deprecated API functions
by Charles R Harris 13 Jul '08

13 Jul '08

The following numpy API functions have been deprecated and will be removed in 1.3 PyArray_FromDims PyArray_FromDimsAndDataAndDescr PyArray_As1D If there are other functions that should be added to the list, let me know. Chuck

1 0

Is PyArray_AsCArray really the best replacement for deprecated PyArray_As1D?
by Charles R Harris 13 Jul '08

13 Jul '08

Just askin'. Neither is mentioned in the numpy book. Chuck

1 0

binary_repr dtype dependence
by Ralf Gommers 13 Jul '08

13 Jul '08

Hi all, binary_repr() behaves differently for different types of ints/uints: In [210]: binary_repr(255) Out[210]: '11111111' In [211]: binary_repr(uint32(255)) Out[211]: '11111111' In [212]: binary_repr(uint16(255)) Out[212]: '1' In [213]: np.__version__ Out[213]: '1.0.4' Is this intended/acceptable behavior? It is due to the use of hex() (checked that it is still present in trunk), which expects int or long int. Cheers, Ralf

2 1

trapz documentation on Examples pages
by Alan G Isaac 13 Jul '08

13 Jul '08

There are two examples pages: http://www.scipy.org/Numpy_Example_List_With_Doc http://www.scipy.org/Numpy_Example_List The latter says: This page contains a large database of examples demonstrating most of the Numpy functionality. Numpy Example List With Doc has these examples interleaved with the built-in documentation, but is not as regularly updated as this page. So I expect the latter to be most complete and up-to-date. (Indeed, I assume it automagically generated.) But ``trapz`` is documented on the former but not the latter. Is there a reason? Thank you, Alan Isaac

2 3

loadtxt and N/A
by Bryan Fodness 13 Jul '08

13 Jul '08

I am using loadtxt and I have missing values that are show up as N/A. I get a, ValueError: invalid literal for float(): N/A Is there a way to ignore these?

2 1

Doctests for extensions/cython code
by Fernando Perez 13 Jul '08

13 Jul '08

Hi all (esp. Alan McIntyre), I'm attaching two little tarballs with a set of tools that may come in handy for testing numpy: - plugin.tgz contains a Nose plugin that works around various issues in the python stdlib, in nose and in cython, to enable the testing of doctests embedded in extension modules. I must note that this code also provides a second plugin that is ipython-aware, and the code as shipped is NOT yet acceptable for public use in numpy, because it insantiates a full ipython on import. But I wrote this primarily for ipython, so for us that's OK. For numpy, we obviously must first remove all the ipython-specific code from there. The two plugins are separated, so it's perfectly doable, I just ran out of time. I'm putting it here in the hopes that it will be useful to Alan, who can strip it of the ipython dependencies and start using it in the numpy tests. The one thing I didn't figure out yet was how to load the plugin from within a python script (instead of doing it at the command line via 'nosetests --extdoctests'). But this should be trivial, it's just a matter of finding the right call in nose, and you may already know it. - primes.tgz is the cython 'primes' example, souped up with trivial code that contains some extra doctests both in python and in extension code. It's just meant to serve as a test for the above plugin (I also used it to provide a self-contained cython example complete with a setup.py file in a seminar, so it can be useful as a starter example for some). I don't know if today's numpy.test() picks up doctests in extension modules (if it does, I'd like to know how). I suspect the answer is not, but if we are to encourage better examples that serve as doctests, then actdually testing them would be good. I hope this helps in that regard. Cheers f

2 1

Schedule for 1.2.0
by Jarrod Millman 13 Jul '08

13 Jul '08

The NumPy 1.2.0 release date (8/24/08) is rapidly approaching and we need everyone's help. David Cournapeau has volunteered to take the lead on coordinating this release. The main focus of this release is on improved documentation and a new testing framework based on nose with only a few new features. The only new features that I am aware of being planned for this release are: * Robert Kern has a function that does efficient, pure-Python broadcasting of arrays. It is quite useful in certain instances where you want to have the broadcasting feature of ufuncs but cannot cast your function into a form where you can make use of ufuncs. * When writing big arrays to filelike objects in the new NPY format introduced in 1.1, it would be good to do so in chunks. Robert de Almeida's arrayterator would do the job well. When Robert K. implemented the NPY format, he asked him if we could use arrayterator in numpy and expose it as a public API in numpy, too. He agreed, but he had some fixes to do and unit tests to write. This has now been completed and arrayterator 1.0 has been released: http://pypi.python.org/pypi/arrayterator/1.0 Another issue that we should address is whether it is OK to postpone the planned API changes to histogram and median. A couple of people have mentioned to me that they would like to delay the API changes to 1.3, which seems reasonable to me. If anyone would prefer that we make the planned API changes for histogram and median in 1.2, please speak now. Here is the schedule for 1.2.0: - 8/05/08 tag the 1.1.1rc1 release and prepare packages - 8/12/08 tag the 1.1.1rc2 release and prepare packages - 8/23/08 tag the 1.1.1 release and prepare packages - 8/24/08 announce release We need to follow this schedule as closely as possible because we should get SciPy 0.7.0 out ASAP (I will send an email out about scipy 0.7.0 tomorrow). Also it would be very good to have release candidates of the newest NumPy and SciPy available for the SciPy 2008 conference. I would also like to get them both out in time for the new school year. If you have any additional features that you have been working on, please let me know ASAP. Otherwise, all feature development should take place on a branch from this point forward. If you want to open a ticket specifically for this bug-fix release, please use the NumPy 1.2.0 milestone: http://scipy.org/scipy/numpy/milestone/1.2.0 Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/

1 1

How to handle a busted API?
by Charles R Harris 13 Jul '08

13 Jul '08

Hi All, This is apropos ticket #805 <http://scipy.org/scipy/numpy/ticket/805>. The reporter wants to change the signature of the functions PyArray_FromDims and PyArray_FromDimsAndDataAndDesc, which we really can't do at this point because they are part of the Numpy API. The problem can be seen in the current signatures: PyArray_FromDims(int nd, int *d, int type) PyArray_FromDimsAndDataAndDescr(int nd, int *d, PyArray_Descr *descr, char *data) where d points to the desired dimensions. On 64 bit architectures the int type can be too small to hold the dimensions. Now these functions are old and retained for compatibility; the user's problem turned up in num_utils, which is a C++ interface to Numeric. Quite possibly other programs use it also (BOOST?). So the question is, how do we go about dealing with this. Do we remove them at some point, breaking compatibility and the current API? If so, when do we do this? Should we issue a deprecation warning? If so, how do we do it from C? Should it show up at run time or compile time? Chuck

2 5

A correction to numpy trapz function
by Nadav Horesh 12 Jul '08

12 Jul '08

The function trapz accepts x axis vector only for axis=-1. Here is my modification (correction?) to let it accept a vector x for integration along any axis: def trapz(y, x=None, dx=1.0, axis=-1): """ Integrate y(x) using samples along the given axis and the composite trapezoidal rule. If x is None, spacing given by dx is assumed. If x is an array, it must have either the dimensions of y, or a vector of length matching the dimension of y along the integration axis. """ y = asarray(y) nd = y.ndim slice1 = [slice(None)]*nd slice2 = [slice(None)]*nd slice1[axis] = slice(1,None) slice2[axis] = slice(None,-1) if x is None: d = dx else: x = asarray(x) if x.ndim == 1: if len(x) != y.shape[axis]: raise ValueError('x length (%d) does not match y axis %d length (%d)' % (len(x), axis, y.shape[axis])) d = diff(x) return tensordot(d, (y[slice1]+y[slice2])/2.0,(0, axis)) d = diff(x, axis=axis) return add.reduce(d * (y[slice1]+y[slice2])/2.0,axis) Nadav.

3 5

Missing NULL return checks?
by Charles R Harris 12 Jul '08

12 Jul '08

PyArray_DescrFromType can return NULL static PyArray_Descr * PyArray_DescrFromType(int type) { PyArray_Descr *ret = NULL; if (type < PyArray_NTYPES) { ret = _builtin_descrs[type]; } else if (type == PyArray_NOTYPE) { /* * This needs to not raise an error so * that PyArray_DescrFromType(PyArray_NOTYPE) * works for backwards-compatible C-API */ return NULL; } else if ((type == PyArray_CHAR) || (type == PyArray_CHARLTR)) { ret = PyArray_DescrNew(_builtin_descrs[PyArray_STRING]); if (ret == NULL) { return NULL; } ret->elsize = 1; ret->type = PyArray_CHARLTR; return ret; } else if (PyTypeNum_ISUSERDEF(type)) { ret = userdescrs[type - PyArray_USERDEF]; } else { int num = PyArray_NTYPES; if (type < _MAX_LETTER) { num = (int) _letter_to_num[type]; } if (num >= PyArray_NTYPES) { ret = NULL; } else { ret = _builtin_descrs[num]; } } if (ret == NULL) { PyErr_SetString(PyExc_ValueError, "Invalid data-type for array"); } else { Py_INCREF(ret); } return ret; } Yet it is unchecked in several places: static int PyArray_CanCastSafely(int fromtype, int totype) { PyArray_Descr *from, *to; register int felsize, telsize; if (fromtype == totype) return 1; if (fromtype == PyArray_BOOL) return 1; if (totype == PyArray_BOOL) return 0; if (totype == PyArray_OBJECT || totype == PyArray_VOID) return 1; if (fromtype == PyArray_OBJECT || fromtype == PyArray_VOID) return 0; from = PyArray_DescrFromType(fromtype); /* * cancastto is a PyArray_NOTYPE terminated C-int-array of types that * the data-type can be cast to safely. */ if (from->f->cancastto) { int *curtype; curtype = from->f->cancastto; while (*curtype != PyArray_NOTYPE) { if (*curtype++ == totype) return 1; } } if (PyTypeNum_ISUSERDEF(totype)) return 0; to = PyArray_DescrFromType(totype); telsize = to->elsize; felsize = from->elsize; Py_DECREF(from); Py_DECREF(to); switch(fromtype) { case PyArray_BYTE: case PyArray_SHORT: case PyArray_INT: case PyArray_LONG: case PyArray_LONGLONG: if (PyTypeNum_ISINTEGER(totype)) { if (PyTypeNum_ISUNSIGNED(totype)) { return 0; } else { return (telsize >= felsize); } } else if (PyTypeNum_ISFLOAT(totype)) { if (felsize < 8) return (telsize > felsize); else return (telsize >= felsize); } else if (PyTypeNum_ISCOMPLEX(totype)) { if (felsize < 8) return ((telsize >> 1) > felsize); else return ((telsize >> 1) >= felsize); } else return totype > fromtype; case PyArray_UBYTE: case PyArray_USHORT: case PyArray_UINT: case PyArray_ULONG: case PyArray_ULONGLONG: if (PyTypeNum_ISINTEGER(totype)) { if (PyTypeNum_ISSIGNED(totype)) { return (telsize > felsize); } else { return (telsize >= felsize); } } else if (PyTypeNum_ISFLOAT(totype)) { if (felsize < 8) return (telsize > felsize); else return (telsize >= felsize); } else if (PyTypeNum_ISCOMPLEX(totype)) { if (felsize < 8) return ((telsize >> 1) > felsize); else return ((telsize >> 1) >= felsize); } else return totype > fromtype; case PyArray_FLOAT: case PyArray_DOUBLE: case PyArray_LONGDOUBLE: if (PyTypeNum_ISCOMPLEX(totype)) return ((telsize >> 1) >= felsize); else return (totype > fromtype); case PyArray_CFLOAT: case PyArray_CDOUBLE: case PyArray_CLONGDOUBLE: return (totype > fromtype); case PyArray_STRING: case PyArray_UNICODE: return (totype > fromtype); default: return 0; } } Furthermore, the last function can fail, but doesn't seem to have an error return. What is the best way to go about cleaning this up? Chuck

2 3

NumPy-Discussion July 2008