large arrays in python (scientific)

Jason Orendorff jason at
Wed Jan 9 20:04:16 EST 2002

> > from __future__ import generators
> > from Scientific.Functions.LeastSquares import leastSquaresFit
> > This reads the data from the file once per iteration
> > (leastSquaresFit is an iterative algorithm), which is slow;
> > but it does not store the whole data set in memory, which
> > might be helpful.
> Assuming that your data points are evenly distributed in a grid,
> you could
> probably trim that down so that you didn't have to explicitly store the
> (x,y,z).

Indeed, ideally:

from __future__ import generators

class MyData:
    """ Wrapper for data.

    This presents a Numeric Python array in a way that
    Scientific.Functions.LeastSquares can understand.

    def __init__(self, array):
        self.array = array

    def __iter__(self):
        array = self.array
        [xr, yr, zr] = [range(N) for N in array.shape]
        for x in xr:
            for y in yr:
                for z in zr:
                    yield ((x, y, z), array[x, y, z])

> BTW- I keep reading interesting examples of generators... they seem
> interesting, and occassionally even useful.  Where can I find some good
> info on 'em?

Well... I don't know.  PEP 255; python22/lib/test/
this article...

## Jason Orendorff

More information about the Python-list mailing list