[Numpy-discussion] Re: Trying out Numeric3

Perry Greenfield perry at stsci.edu
Wed Mar 23 19:28:49 EST 2005


> Scott Gilbert wrote:
>
> > --- Michiel Jan Laurens de Hoon <mdehoon at ims.u-tokyo.ac.jp> wrote:
> >
> >>>>Do 4 gigabyte 1D numerical python arrays occur in practice?
> >>
> >>Why? There needs to be a good reason to break compatibility. Who needs
> >>this?
> >>
> >
> > I (and others I work with) routinely deal with 1D datasets that are
> > multiple gigabytes in length.  Working with terabyte datasets
> is on my near
> > horizon.
>
> I see. Then I agree, we need to fix the dimensions and strides in
> PyArrayObject.
> Thanks, Scott.
>
I'll also add that we've already had internal requests to deal with files
that
large as well as external queries about support large files. Believe me,
files of this size are becoming much more common than you realize.

Perry






More information about the NumPy-Discussion mailing list