[Tutor] Problems creating very large arrays (over 10 million indices)

[Reggie Dugard]

Sorry, I was a little too fast on the send button on that last message. 
The first example should read:

a = Numeric.fromstring(fp.read(), Numeric.Int16)
a = Numeric.reshape(a, (xsize, ysize, zsize))


On Tue, 2003-07-01 at 12:15, Reggie Dugard wrote:
> Edmund,
> 
> Assuming that fp is a file object opened for reading your file and that
> you've already read past the header, maybe one of these will work for
> you:
> 
> a = Numeric.fromstring(fp.read(), Numeric.Int16)
> Numeric.reshape(a, (xsize, ysize, zsize))
> 
> OR
> 
> a = numarray.fromfile(fp, numarray.Int16, (xsize, ysize, zsize))
> 
> I don't have much experience reading in data from outside sources, so I
> can't say that I've tested these, but I hope they steer you in the right
> direction.
> 
> 
> On Tue, 2003-07-01 at 11:20, DORSEY_EDMUND_K@LILLY.COM wrote:
> > I tried creating a Numeric array but I couldn't get the binary data into 
> > it. The array module was so great cause I could just use readfile to read 
> > in the entire file at once.
> > 
> > I tried this ...
> > 
> > a = Numeric.ones((xsize, ysize, zsize), Numeric.Int16) #create 3d numeric 
> > array of type short filled with ones (placeholders)
> > a[0][0][0] = file.read(2) #read in two bytes and stick it in the array
> > 
> > Of course it says the that the array 'a' needs an int as input.  Plus if I 
> > do this method I need to use 3 nested for loops and it is significantly 
> > slower than using a normal 1d array.  Is this how I would have to do it 
> > with the Numeric module?  Thanks for all the input you've given. 
> > 
> > ~Edmund
> > 
> > 
> > 
> > 
> > 
> > Edmund,
> > 
> > Given the size of your arrays and that they are 3 dimensional, you may 
> > want
> > to take a look at the Numeric package or the relatively new numarray
> > package.  They can handle multi-dimensional data in a very efficient 
> > manner.
> > They both can be found on sourceforge at 
> > 
> > http://sourceforge.net/projects/numpy/
> > 
-- 
Reggie