Hi Travis, Could you look at bug [ 635104 ] segfault unpickling Numeric 'O' array [ 567796 ] unpickling of 'O' arrays causes segfault (duplicate of previous one) I proposed a (rather simple) solution that I put in the comment of bug [ 635104 ]. But apparently, nobody is looking at those bugs...
I'd like to release a Numeric 24.0 to get the array interface out there. There are also some other bug fixes in Numeric 24.0
Here is the list so far from Numeric 23.7
[Greenfield] Changed so a[0,0] and a[0][0] returns same type when a is 2-d of Int16
This is quite disturbing. In fact for all types that are not exactly equivalent to python type, indexing a multidimensional array (rank > 1) return arrays even if the final shape is (). So type(zeros((5,2,4), Int8 )[0,0,0]) => <type 'array'> type(zeros((5,2,4), Int32 )[0,0,0]) => <type 'array'> type(zeros((5,2), Float32 )[0,0]) => <type 'array'> But type(zeros((5,2,4), Int )[0,0,0]) => <type 'int'> type(zeros((5,2,4), Float64)[0,0,0]) => <type 'float'> type(zeros((5,2,4), Float)[0,0,0]) => <type 'float'> type(zeros((5,2,4), PyObject)[0,0,0]) => <type 'int'> Notice too the weird difference betweeb Int <> Int32 and Float == Float64. However, when indexing a onedimensional array (rank == 1), then we get back scalar for indexing operations on all types. So, when you say "return the same type", do you think scalar or array (it smells like a recent discussion on Numeric3 ...) ?
[unreported] Added array interface [unreported] Allow Long Integers to be used in slices [1123145] Handle mu==0.0 appropiately in ranlib/ignpoi. [unreported] Return error info in ranlib instead of printing it to stderr [1151892] dot() would quit python with zero-sized arrays when using dotblas. The BLAS routines *gemv and *gemm need LDA >= 1. [unreported] Fixed empty for Object arrays
Version 23.8 March 2005 [Cooke] Fixed more 64-bit issues (patch 117603) [unreported] Changed arrayfnsmodule back to PyArray_INT where the code typecasts to (int *). Changed CanCastSafely to check if sizeof(long) == sizeof(int)
I'll wait a little bit to allow last minute bug fixes to go in, but I'd realy like to see this release get out there. For users of Numeric >23.7 try Numeric.empty((10,20),'O') if you want to see an *interesting* bug that is fixed in CVS.
-Travis
On Wed, Apr 06, 2005 at 07:33:56AM +0000, S�bastien de Menten wrote:
Hi Travis,
Could you look at bug [ 635104 ] segfault unpickling Numeric 'O' array [ 567796 ] unpickling of 'O' arrays causes segfault (duplicate of previous one)
I proposed a (rather simple) solution that I put in the comment of bug [ 635104 ]. But apparently, nobody is looking at those bugs...
This is too true. Travis added myself and Michiel de Hoon recently to the developers, so there's some new blood, and we've been banging on things, though. I'll have a look at it if I've got time. I personally really hate bugs that crash my interpreter :-)
I'd like to release a Numeric 24.0 to get the array interface out there. There are also some other bug fixes in Numeric 24.0
Here is the list so far from Numeric 23.7
[Greenfield] Changed so a[0,0] and a[0][0] returns same type when a is 2-d of Int16
This is quite disturbing. In fact for all types that are not exactly equivalent to python type, indexing a multidimensional array (rank > 1) return arrays even if the final shape is (). So type(zeros((5,2,4), Int8 )[0,0,0]) => <type 'array'> type(zeros((5,2,4), Int32 )[0,0,0]) => <type 'array'> type(zeros((5,2), Float32 )[0,0]) => <type 'array'> But type(zeros((5,2,4), Int )[0,0,0]) => <type 'int'> type(zeros((5,2,4), Float64)[0,0,0]) => <type 'float'> type(zeros((5,2,4), Float)[0,0,0]) => <type 'float'> type(zeros((5,2,4), PyObject)[0,0,0]) => <type 'int'>
Notice too the weird difference betweeb Int <> Int32 and Float == Float64.
That's because Int is *not* Int32. Int32 is the first typecode of '1sil' that has 32 bits. For (all?) platforms I've seen, that'll be 'i'. Int corresponds to a Python integer, and Float corresponds to a Python float. Now, a Python integer is actually a C long, and a Python float is actually a C double. I've made a table: Numeric type typecode Python type C type Array type Int 'l' int long PyArray_LONG Int32 'i' [1] N/A int PyArray_INT Float 'd' float double PyArray_DOUBLE Float32 'f' N/A float PyArray_FLOAT Float64 'd' float double PyArray_DOUBLE [1] assuming sizeof(int)==4, which is true on most platforms. There are some 64-bit platforms where this won't be true, I think. On (all? most?) 32-bit platforms, sizeof(int) == sizeof(long) == 4, so both Int and Int32 be 32-bit quantities. Not so on some 64-bit platforms (Linux on an Athlon 64, like the one I'm typing at now), where sizeof(long) == 8. I've been fixing oodles of assumptions in Numeric where ints and longs have been used interchangeably, hence the extended discussion :-) [I haven't addressed here why you get an array sometimes and a Python type the others. This is the standard, old, behaviour -- it's likely not going to change in Numeric. Whether it's a *good* thing is another question. scipy.base and numarray do it differently.] -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca
Sébastien de Menten wrote:
Hi Travis,
Could you look at bug [ 635104 ] segfault unpickling Numeric 'O' array [ 567796 ] unpickling of 'O' arrays causes segfault (duplicate of previous one)
I proposed a (rather simple) solution that I put in the comment of bug [ 635104 ]. But apparently, nobody is looking at those bugs...
One thing I don't like about sourceforge bug tracker is that I don't get any email notification of bugs. Is there an option for that? I check my email, far more often than I check a website. Sourceforge can be quite slow to manipulate around in. Now, that you've mentioned it, I'll look into it. I'm not sure that object arrays could every be pickled correctly. -Travis
I'd like to release a Numeric 24.0 to get the array interface out there. There are also some other bug fixes in Numeric 24.0
Here is the list so far from Numeric 23.7
[Greenfield] Changed so a[0,0] and a[0][0] returns same type when a is 2-d of Int16
This is quite disturbing. In fact for all types that are not exactly equivalent to python type, indexing a multidimensional array (rank > 1) return arrays even if the final shape is ().
So, what should it do? This is the crux of a long-standing wart in Numerical Python that nobody has had a good solution to (I think the array scalars that have been introduced for scipy.base are the best solution yet). Right now, the point is that different things are done for different indexing strategies. Is this a good thing? Maybe it is. We can certainly leave it the way it is now and back-out the change. The current behavior is: Subscripting always produces a rank-0 array if the type doesn't match a basic Python type. Item getting always produces a basic Python type (even if there is no match). So a[0,0] and a[0][0] will return different things if a is an array of short's for example. This may be what we live with and just call it a "feature"
So type(zeros((5,2,4), Int8 )[0,0,0]) => <type 'array'> type(zeros((5,2,4), Int32 )[0,0,0]) => <type 'array'> type(zeros((5,2), Float32 )[0,0]) => <type 'array'> But type(zeros((5,2,4), Int )[0,0,0]) => <type 'int'> type(zeros((5,2,4), Float64)[0,0,0]) => <type 'float'> type(zeros((5,2,4), Float)[0,0,0]) => <type 'float'> type(zeros((5,2,4), PyObject)[0,0,0]) => <type 'int'>
Notice too the weird difference betweeb Int <> Int32 and Float == Float64.
This has been in Numeric for a long time (the coercion problems was one of the big reasons for it). If you return a Python integer when indexing an Int8 array then use that for multiplication you get undesired up-casting. There is no scalar Int8 type to return (thus a 0-dimensional array that can act like a scalar is returned). In scipy.base there are now scalar-like objects for all of the supported array types which is one solution to this problem that was made possible by the ability to inherit in C that is now part of Python. What platform are you on? Notice that Int is interpreted as C-long (PyArray_LONG) while Int32 is PyArray_INT. This has been another wart in Numerical Python. By the way, I've fixed PyArray_Return so that if sizeof(long)==sizeof(int) then PyArray_INT also returns a Python integer. I think for places where sizeof(long)==sizeof(int) PyArray_LONG and PyArray_INT should be treated identically.
However, when indexing a onedimensional array (rank == 1), then we get back scalar for indexing operations on all types.
So, when you say "return the same type", do you think scalar or array (it smells like a recent discussion on Numeric3 ...) ?
I just think the behavior ought to be the same for a[0,0] or a[0][0] but maybe I'm wrong and we should keep the dichotomy to satisfy both groups of people. Because of the problems I alluded to, sometimes a 0-dimensional array should be returned. -Travis
Travis Oliphant <oliphant@ee.byu.edu> writes:
Sébastien de Menten wrote:
Hi Travis,
Could you look at bug [ 635104 ] segfault unpickling Numeric 'O' array [ 567796 ] unpickling of 'O' arrays causes segfault (duplicate of previous one)
I proposed a (rather simple) solution that I put in the comment of bug [ 635104 ]. But apparently, nobody is looking at those bugs...
One thing I don't like about sourceforge bug tracker is that I don't get any email notification of bugs. Is there an option for that? I check my email, far more often than I check a website. Sourceforge can be quite slow to manipulate around in.
I think if the bug is assigned to you, you get email.
So type(zeros((5,2,4), Int8 )[0,0,0]) => <type 'array'> type(zeros((5,2,4), Int32 )[0,0,0]) => <type 'array'> type(zeros((5,2), Float32 )[0,0]) => <type 'array'> But type(zeros((5,2,4), Int )[0,0,0]) => <type 'int'> type(zeros((5,2,4), Float64)[0,0,0]) => <type 'float'> type(zeros((5,2,4), Float)[0,0,0]) => <type 'float'> type(zeros((5,2,4), PyObject)[0,0,0]) => <type 'int'>
Notice too the weird difference betweeb Int <> Int32 and Float == Float64.
By the way, I've fixed PyArray_Return so that if sizeof(long)==sizeof(int) then PyArray_INT also returns a Python integer. I think for places where sizeof(long)==sizeof(int) PyArray_LONG and PyArray_INT should be treated identically.
I don't think this is good -- it's just papering over the problem. It leads to different behaviour on machines where sizeof(long) != sizeof(int) (specifically, the problem reported by Nils Wagner *won't* be fixed by this on my machine). On some machines x[0] will give you a int (where x is an array of Int32), on others an array: not fun. I see you already beat me in changing PyArray_PyIntAsInt to support rank-0 integer arrays. How about changing that to instead using anything that int() can handle (using PyNumber_AsInt)? This would include anything int-like (rank-0 integer arrays, scipy.base array scalars, etc.). The side-effect is that you can index using floats (since int() of a float truncates it towards 0). If this is a big deal, I can special-case floats to raise an error. This would make (almost) all Numeric behaviour consistent with regards to using Python ints, Python longs, and rank-0 integer arrays, and other int-like objects.
However, when indexing a onedimensional array (rank == 1), then we get back scalar for indexing operations on all types.
So, when you say "return the same type", do you think scalar or array (it smells like a recent discussion on Numeric3 ...) ?
I just think the behavior ought to be the same for a[0,0] or a[0][0] but maybe I'm wrong and we should keep the dichotomy to satisfy both groups of people. Because of the problems I alluded to, sometimes a 0-dimensional array should be returned.
I'd prefer having a[0,0] and a[0][0] return the same thing: it's not the special case of how to do two indices: it's the special-casing of rank-1 arrays as compared to rank-n arrays. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca
participants (4)
-
cookedm＠physics.mcmaster.ca -
David M. Cooke
-
S�bastien de Menten -
Travis Oliphant