Mailman 3 December 2002 - NumPy-Discussion

Non-numerical info associated with sub-arrays
by Magnus Lie Hetland 28 Dec '02

28 Dec '02

I'm working on some two-dimensional tables of data, where some data are numerical, while other aren't. I'd like to use numarray's numerical capabilities with the numerical parts (columns) while keeping the data in each row together. (I'm sure this generalizes to more dimensions, and to sub-arrays in general, not just rows.) It's not a hard problem, really, but the obvious solution--to keep the other rows in separate arrays/lists and just juggle things around--seems a bit clunky. I was just wondering if anyone had other ideas (would it be practical to include all the data in a single array somehow--I seem to recall that Numeric could have arbitrary object arrays, but I'm not sure whether numarray supports this?) or perhaps some hints on how to organize code around this? I wrote a small class that wraps things up and works a bit lik R/S-plus's data frames; is there some other more standard code out there for this sort of thing? (It's a problem that crops up often in data processing of various kinds...) Thanks, Magnus -- Magnus Lie Hetland http://hetland.org

4 10

Increased polymorphism?
by Magnus Lie Hetland 28 Dec '02

28 Dec '02

I know PyObject arrays aren't supported in numarray yet -- but how about making the functions more polymorphic? Yes, they work on other sequence types such as lists -- but there seems to be quite a bit of type checking going on. I suppose that is necessary for performance reasons, but how about some polymorphic behaviour in the cases where an unknown type is encountered? It would be nice, for example, if sum() worked on general iterators and generators, and if argsort() worked on, e.g., lists of strings... This would make it easier to integrate the purely numeric stuff with other parts of Python programs... It may be that I'm going beyond the scope of numarray here, but this sort of agnostic signature-based polymorphism is very Pythonic. -- Magnus Lie Hetland http://hetland.org

1 0

from RandomArray2 import *?
by Magnus Lie Hetland 28 Dec '02

28 Dec '02

I just tried a starred import from RandomArray2, but it seems that it didn't work properly -- e.g. the uniform function wasn't imported. And I can't see any __all__ attribute; what's going on? -- Magnus Lie Hetland http://hetland.org

2 1

Further comments
by Edward C. Jones 28 Dec '02

28 Dec '02

Should the variable "type" be used in numarray? It is an important function in Python. --------------------------------------- There needs to be a function or method that returns the number of elements in an array. def Elements(array): """Number of elements in an array. This version is slow. """ return numarray.multiply.reduce(array.shape) --------------------------------------- I write code using both PIL and numarray. PIL uses strings for modes and numarray uses (optionally) strings as typecodes. This causes problems. One fix is to emit a DeprecationWarning when string typecodes are used. Two functions are needed: StringTypeWarningOn and StringTypeWarningOff. The default should be to ignore this warning. In my code I use the following workarounds: def SameType(x, y): """Are the two input the same object of NumericType?""" if isinstance(x, NumericType) and isinstance(y, NumericType) \ and x == y: return True return False def IsTypeInList(typecode, seq): """Is a NumericType object in a list of NumericType objects?""" if not isinstance(typecode, NumericType): return False for item in seq: if isinstance(item, NumericType) and typecode == item: return True return False --------------------------------------- The following function is useful for downsizing arrays. I suggest that it should be a ufunc method. This is how I have used reduceat in Numeric. def blockreduce(array, blocksizes, ufunc): """Apply ufunc.reduce to blocks in an array.""" dims = len(array.shape) if type(blocksizes) is IntType: blocksizes = dims * [blocksizes] if len(blocksizes) != dims: raise TypeError, 'blocksizes must be same length as shape' for i in range(dims): if array.shape[i] % blocksizes[i] != 0: raise ValueError, 'blocksizes must exactly divide ' \ 'the corresponding array dimension' for i in range(dims): array = array.copy() newshape = (array.shape[0] / blocksizes[i], blocksizes[i]) + \ array.shape[1:] array.shape = newshape array = ufunc.reduce(array, 1) dims = len(array.shape) # (0,1,2,3) --> (1,2,3,0) perm = tuple(range(1, dims)) + (0,) array = numarray.transpose(array, perm) return array

3 2

"<<", ">>" undocumented
by Edward C. Jones 22 Dec '02

22 Dec '02

The operators "<<" and ">>" seem to be undocumented. Is there a reason for this? Ed Jones

1 0

bug in Numeric, LinearAlgebra
by Cong 22 Dec '02

22 Dec '02

I've just upgraded from Numeric-21.0 to Numeric-22.0 ( python-2.2.2 ) and found a bug. Do anyone use LinearAlgebra too ? Now, I downed it to Numeric 21.0 . This version works very well. Cong $ python Python 2.2.2 (#1, Dec 14 2002, 05:45:37) [GCC 3.2.1] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import LinearAlgebra Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib/python2.2/site-packages/Numeric/LinearAlgebra.py", line 10, in ? import MLab File "/usr/lib/python2.2/site-packages/Numeric/MLab.py", line 20, in ? import RandomArray File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 30, in ? seed() File "/usr/lib/python2.2/site-packages/Numeric/RandomArray.py", line 24, in seed ndigits = int(math.log10(t)) OverflowError: float too large to convert

1 1

howto build numarray 0.4 ? file 'Src/_convmodule.c' does not exist
by Sebastian Haase 19 Dec '02

19 Dec '02

Hi all, I downloaded numarray 0.4 about 5 minutes after I got the announce but my naive 'python2.2 ./setup.py build' gives this haase@baboon:~/numarray-0.4: python2.2 ./setup.py build running build running build_py not copying Lib/ndarray.py (output up-to-date) not copying Lib/numtest.py (output up-to-date) not copying Lib/codegenerator.py (output up-to-date) not copying Lib/ufunc.py (output up-to-date) not copying Lib/testdata.py (output up-to-date) not copying Lib/numarray.py (output up-to-date) not copying Lib/ieeespecial.py (output up-to-date) not copying Lib/recarray.py (output up-to-date) not copying Lib/template.py (output up-to-date) not copying Lib/arrayprint.py (output up-to-date) not copying Lib/typeconv.py (output up-to-date) not copying Lib/numinclude.py (output up-to-date) not copying Lib/numarrayext.py (output up-to-date) not copying Lib/_ufunc.py (output up-to-date) not copying Lib/chararray.py (output up-to-date) not copying Lib/numtestall.py (output up-to-date) not copying Lib/numerictypes.py (output up-to-date) not copying Lib/memmap.py (output up-to-date) running build_ext building '_conv' extension error: file 'Src/_convmodule.c' does not exist What am I missing? I'm running Linux (debian woody) on i386. Thanks, Sebastian

3 4

Comments on numarray - 4
by Edward C. Jones 18 Dec '02

18 Dec '02

The optional shape argument to fromstring is not documented. ----------------- I suggest a documentation section on NumericTypes. "Int8.bytes" and need documenting. ---------------- Table of Automatic Coercions (found by experiment) B I8 U8 I16 U16 I32 U32 I64 U64 F32 F64 C32 C64 ------------------------------------------------------------------ B | *I8 I8 U8 I16 U16 I32 U32 I64 U64 F32 F64 C32 C64 I8 | I8 *I16 I16 U16 I32 U32 I64 U64 F32 F64 C32 C64 U8 | U8 I16 U16 I32 U32 I64 U64 F32 F64 C32 C64 I16 | I16 *I32 I32 U32 I64 U64 F32 F64 C32 C64 U16 | U16 I32 U32 I64 U64 F32 F64 C32 C64 I32 | I32 *I64 I64 U64 F32 F64 C32 C64 U32 | U32 I64 U64 F32 F64 C32 C64 I64 | I64 *I64 ?F32 F64 C32 C64 U64 | U64 ?F32 F64 C32 C64 F32 | F32 F64 C32 C64 F64 | F64 ?C32 C64 C32 | C32 C64 C64 | C64 The names are listed in the order used in "numerictypes.py". The "*" cases are exceptions treated there. Are the "?" conversions correct? See "genericPromotionExclusions" in "numerictypes.py".

2 1

Re: [Numpy-discussion] Index array confusion...
by Magnus Lie Hetland 16 Dec '02

16 Dec '02

Todd Miller <jmiller(a)stsci.edu>: > [snip] > What I'm saying is that in the future it will work correctly. :) OK :) > Most likely as you suggest, but I haven't really looked at the code, > just the rather sickening results. Well, what I was asking about was really what you meant by "correct". My interpretation of "correct" here was that only tuples and slices would be allowed as indexes. BTW: I found something else that I think is rather odd: >>> x = [0, 0, 0] >>> y = [1, 1, 1] >>> p = 1 >>> x[p:], y[p:] = y[p:], x[p:] >>> x, y ([0, 1, 1], [1, 0, 0]) >>> x = array([0, 0, 0]) >>> y = array([1, 1, 1]) >>> x[p:], y[p:] = y[p:], x[p:] >>> x, y (array([0, 1, 1]), array([1, 1, 1])) This seems like a bug to me. The assignment ought to swap the tails of the sequences, as is the case with lists, but with numeric arrays, some weird form of overwriting occurs. I guess this may be an optimization (i.e. to avoid making copies), but it is rather confusing. What do you think? > Todd -- Magnus Lie Hetland Practical Python The Anygui Project http://hetland.org http://ppython.com http://anygui.org

3 7

Re: numarray questions
by Todd Miller 16 Dec '02

16 Dec '02

Mika Illouz wrote: >Hi Todd, > > > >>Thanks for the encouragement! numarray-0.4 is coming out today... >> >> > >Finally got around to playing with version 0.4 and the new NA_New (code >attached). Wnen I ran the test, I got the following trace: > > > >>>>import _test_numarray >>>>z = _test_numarray.t1() >>>> >>>> >Traceback (most recent call last): > File "<stdin>", line 1, in ? > File "/usr/lib/python2.2/site-packages/numarray/numarray.py", line 432, in __init__ > bytestride=bytestride) > File "/usr/lib/python2.2/site-packages/numarray/ndarray.py", line 173, in __init__ > self._data = memory.new_memory(size) >OverflowError: long int too large to convert to int > >What am I doing wrong? > The parameter order in the call in t1() should be: NA_New(a, tInt32, 1, length) Because it is: NA_New(a, tInt32, length, 1) numarray thinks you're declaring a 5 dimensional array with some bizarre dimensions from "undefined values" on the stack. One of these dimensions appears to be > MAX_INT. >Thanks again, >Mika > > Regards, Todd > > >------------------------------------------------------------------------ > >#include <Python.h> >#include <libnumarray.h> > >static PyObject* t1( PyObject* self, PyObject* args ) >{ > int length = 5; > int* a = new int[ length ]; > for ( int i = 0 ; i < length ; i++ ) { > a[i] = i; > } > > PyArrayObject* vec = NA_New( a, tInt32, length, 1 ); > return (PyObject*)vec; > // return NA_ReturnOutput( NULL, vec ); > >} > >static PyMethodDef TestNumarrayMethods[] = { > {"t1", > t1, > METH_VARARGS, > "first test" > }, > { NULL, NULL, 0, NULL} >}; > >extern "C" void init_test_numarray() >{ > Py_InitModule( "_test_numarray", TestNumarrayMethods ); > import_libnumarray(); >} > >

1 0