[Numpy-discussion] Suggested Addition to the new array PEP

27 Jul 2005

      Hi all, but particularly Travis,

I just took the time to read the new array PEP:

http://numeric.scipy.org/PEP.txt

It looks great.

I do have one suggestion for an addition method:

change_type(NewType) # or some other name:

Change the type that the bytes in the array represent, without changing 
the data at. Dimensions will change if the new type has a different 
itemsize than the old.

This would be the equivalent of:

A = Numeric.fromstring(A.tostring(), NewType)

but without the extra copies of the data.

When would anyone want this? I've written a function that does exactly 
this with Numeric arrays in order to efficiently slice, dice and 
re-arange a bunch of binary data files.

The files consist of a bunch of C structs written sequentially to a 
file. In this case the structs are: (long, long, char)

I have a file which is N of these structs written sequentially to a 
file, I can do:

NumBytes = 9 # number of bytes in a record
# Read the file
data = fromstring(file.read(NumBytes*NumTimesteps*NumLEs),UnsignedInt8)
file.close()

data.shape = (NumTimesteps,NumLEs,NumBytes)
# extract LE data:
LEs = data[:,:,:8].copy()
# extract the flags
flags = data[:,:,8].copy()
del data

# here I can create the Int32 array without making a copy:
# changetype returns a rank-1 array
changetype(LEs,Int32)

# It now gets reshaped to fit what it represents.
LEs.shape = (NumTimesteps,NumLEs,2)

I think it would only work in the general case with a contiguous array. 
All this really requires is to re-set the attributes of the array. with 
Numeric, the code is (for contiguous arrays):

     NewDescription = PyArray_DescrFromType(typecode);
     NewElementSize = NewDescription->elsize;

     if (TotalBytes % NewElementSize > 0){
       PyErr_SetString (PyExc_ValueError,
                        "The input array is the wrong size for the 
requested type");
       return NULL;
     }

     Array->descr = NewDescription;
     // does the old descr need to be freed?? how??

     Array->nd = 1;
     Array->strides[0] = NewElementSize;
     Array->dimensions[0] = TotalBytes / NewElementSize; The core

By the way, I'm also not seeing tostring() or tobuffer() listed in the PEP

-Chris

-- 
Christopher Barker, Ph.D.
Oceanographer

NOAA/OR&R/HAZMAT         (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker@noaa.gov

[Numpy-discussion] Suggested Addition to the new array PEP

Chris Barker