Re-implementation of Python Numerical arrays (Numeric) available for download
We have been working on a reimplementation of Numeric, the numeric array manipulation extension module for Python. The reimplementation is virtually a complete rewrite and because it is not completely backwards compatible with Numeric, we have dubbed it numarray to prevent confusion. While we think this version is not yet mature enough for most to use in everyday projects, we are interested in feedback on the user interface and the open issues (see the documents on the web page shown below). We also welcome those who would like to contribute to this effort by helping with the development or adding libraries. An early beta version is available on sourceforge as the package Numarray (http://sourceforge.net/projects/numpy/) Information on the goals, changes in user interface, open issues, and design can be found at http://aten.stsci.edu/numarray
Perry Greenfield wrote:
you ask a few questions on the information website, here are some of my answers for things i "care" about. note that my main use of numpy is as a pixel buffer for images. some of the changes like avoiding type promotion sounds really good to me :] 5) should the implementation be bulletproof for private vars? i don't think you should worry about this. as long as the interface is well defined, i wouldn't worry about protecting users from themselves. i this it will be the rare numarray user will be in a situation where they need to modify the internal C data. 7) necessary to add other types? yes. i really want unsigned int16 and unsigned int32. all my operations are on pixel data, and things can just get messy when i need to treat packed color values as signed integers. 8) negative and out-of-range indices? i'd prefer them to be kept as similar to python as can be. the current implementation in Numeric is nice for me. one other thing i'd like there to be a little focus on is adding my own new ufunc operators. for image manipulation i'd like new ufunc operators that clamp the results to legal values. i'd be happy to do this myself, but i don't believe it's possible with the current Numeric. the last thing i really really want is for this to be rolled into the standard python distribution. that is perhaps the most important aspect for me. i do not like requiring the extra dependency for generic numeric arrays. :]
On Saturday 17 November 2001 00:11 am, you wrote:
note that my main use of numpy is as a pixel buffer for images. some of the changes like avoiding type promotion sounds really good to me :]
I have exactly the same application so I agree with this.
Yes please! One of the things that irritates me most on the original Numeric is that some types are lacking. I think the whole range of data types should be supported, even if some may be seldom used by most people.
I write functions in C that directly access the numeric data. I don't use the ufunc api. One reason that I do that is that I want my libary of routines to be useful independent of Numeric, so I only have a tiny glue between my C routines and Numeric. I hope that it will be still possible to do this in the new version.
I second that! Cheers, Peter -- Dr. Peter J. Verveer Bastiaens Group Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer@embl-heidelberg.de
What I've seen looks great. You've all done some good work here. Of course, I do have some feedback. I haven't looked at everything, these points have just caught my eye. Complex Types: ============== 1) I don't like the idea of complex types being a separate subclass of ndarray. This makes them "different." Unless this "difference" can be completely hidden (which I doubt), I would prefer complex types to be on the same level as other numeric types. 2) Also, in your C-API, you have a different pointer to the imaginary data. I much prefer the way it is done currently to have complex numbers represented as an 8-byte, or 16-byte chunk of contiguous memory. Index Arrays: =========== 1) For what it's worth, my initial reaction to your indexing scheme is negative. I would prefer that if a = [[1,2,3,4], [5,6,7,8], [9,10,11,12], [13,14,15,16]] then a[[1,3],[0,3]] returns the sub-matrix: [[ 4, 6], [ 12, 14] i.e. the cross-product of [1,3] x [0,3] This is the way MATLAB works. I'm not sure what IDL does. If I understand your scheme, right now, then I would have to append an extra dimension to my indexing arrays to get this behavior, right? 2) I would like to be able to index the array in a flattenned sense as well (is that possible?) in other words, it would be nice if a[flat(9,10,11)] or something got me the elements 9,10,11 in a one-dimensional interpretation of the array. 3) Why can't you combine slice notation and indexing? Just interpret the slice as index array that would be created from using tha range operator on the same start, stop, and step objects. Is this the plan? That's all for now. I don't mean to be critical, I'm really impressed with what works so far. These are just some concerns I have right now. -Travis Oliphant
Thanks for all of your work. Things seem to be shaping up nicely. I just wanted to second some of the concerns below:
The second comment above is really critical for accessing utility available in a very large number of numerical libraries. In my view this would "break" the utility of numpy severely -- recopying arrays both on the way out and the way in would be extremely cumbersome. -Todd Alan Pitts
Perry Greenfield wrote:
6) Should array properties be accessible as public attributes instead of through accessor methods? We don't currently allow public array attributes to make the Python code simpler and faster (otherwise we will be forced to use __setattr__ and such). This results in incompatibilty with previous code that uses such attributes. I prefer the use of public attributes over accessor methods. -- Paul Barrett, PhD Space Telescope Science Institute Phone: 410-338-4475 ESS/Science Software Group FAX: 410-338-4767 Baltimore, MD 21218
Perry Greenfield wrote:
you ask a few questions on the information website, here are some of my answers for things i "care" about. note that my main use of numpy is as a pixel buffer for images. some of the changes like avoiding type promotion sounds really good to me :] 5) should the implementation be bulletproof for private vars? i don't think you should worry about this. as long as the interface is well defined, i wouldn't worry about protecting users from themselves. i this it will be the rare numarray user will be in a situation where they need to modify the internal C data. 7) necessary to add other types? yes. i really want unsigned int16 and unsigned int32. all my operations are on pixel data, and things can just get messy when i need to treat packed color values as signed integers. 8) negative and out-of-range indices? i'd prefer them to be kept as similar to python as can be. the current implementation in Numeric is nice for me. one other thing i'd like there to be a little focus on is adding my own new ufunc operators. for image manipulation i'd like new ufunc operators that clamp the results to legal values. i'd be happy to do this myself, but i don't believe it's possible with the current Numeric. the last thing i really really want is for this to be rolled into the standard python distribution. that is perhaps the most important aspect for me. i do not like requiring the extra dependency for generic numeric arrays. :]
On Saturday 17 November 2001 00:11 am, you wrote:
note that my main use of numpy is as a pixel buffer for images. some of the changes like avoiding type promotion sounds really good to me :]
I have exactly the same application so I agree with this.
Yes please! One of the things that irritates me most on the original Numeric is that some types are lacking. I think the whole range of data types should be supported, even if some may be seldom used by most people.
I write functions in C that directly access the numeric data. I don't use the ufunc api. One reason that I do that is that I want my libary of routines to be useful independent of Numeric, so I only have a tiny glue between my C routines and Numeric. I hope that it will be still possible to do this in the new version.
I second that! Cheers, Peter -- Dr. Peter J. Verveer Bastiaens Group Cell Biology and Cell Biophysics Programme EMBL Meyerhofstrasse 1 D-69117 Heidelberg Germany Tel. : +49 6221 387245 Fax : +49 6221 387242 Email: Peter.Verveer@embl-heidelberg.de
What I've seen looks great. You've all done some good work here. Of course, I do have some feedback. I haven't looked at everything, these points have just caught my eye. Complex Types: ============== 1) I don't like the idea of complex types being a separate subclass of ndarray. This makes them "different." Unless this "difference" can be completely hidden (which I doubt), I would prefer complex types to be on the same level as other numeric types. 2) Also, in your C-API, you have a different pointer to the imaginary data. I much prefer the way it is done currently to have complex numbers represented as an 8-byte, or 16-byte chunk of contiguous memory. Index Arrays: =========== 1) For what it's worth, my initial reaction to your indexing scheme is negative. I would prefer that if a = [[1,2,3,4], [5,6,7,8], [9,10,11,12], [13,14,15,16]] then a[[1,3],[0,3]] returns the sub-matrix: [[ 4, 6], [ 12, 14] i.e. the cross-product of [1,3] x [0,3] This is the way MATLAB works. I'm not sure what IDL does. If I understand your scheme, right now, then I would have to append an extra dimension to my indexing arrays to get this behavior, right? 2) I would like to be able to index the array in a flattenned sense as well (is that possible?) in other words, it would be nice if a[flat(9,10,11)] or something got me the elements 9,10,11 in a one-dimensional interpretation of the array. 3) Why can't you combine slice notation and indexing? Just interpret the slice as index array that would be created from using tha range operator on the same start, stop, and step objects. Is this the plan? That's all for now. I don't mean to be critical, I'm really impressed with what works so far. These are just some concerns I have right now. -Travis Oliphant
Thanks for all of your work. Things seem to be shaping up nicely. I just wanted to second some of the concerns below:
The second comment above is really critical for accessing utility available in a very large number of numerical libraries. In my view this would "break" the utility of numpy severely -- recopying arrays both on the way out and the way in would be extremely cumbersome. -Todd Alan Pitts
Perry Greenfield wrote:
6) Should array properties be accessible as public attributes instead of through accessor methods? We don't currently allow public array attributes to make the Python code simpler and faster (otherwise we will be forced to use __setattr__ and such). This results in incompatibilty with previous code that uses such attributes. I prefer the use of public attributes over accessor methods. -- Paul Barrett, PhD Space Telescope Science Institute Phone: 410-338-4475 ESS/Science Software Group FAX: 410-338-4767 Baltimore, MD 21218
participants (7)
-
Paul Barrett -
Perry Greenfield -
Pete Shinners -
Peter Verveer -
Tim Hochberg -
Todd Alan Pitts, Ph.D. -
Travis Oliphant