Re: [Numpy-discussion] 2 i/o questions and an announcement

Janko Hauser <jhauser@ifm.uni-kiel.de> writes:
From my understanding of the code, Numpy2 uses the buffer interface. This would be the basis for a fast exchange of data between PIL and Numpy arrays. Regarding IO the pickling of arrays is quite fast. Have you tried this already?
Pickling is fine if all you want to do is create data within Numeric and save it. But it's not useful either for reading data that already exists in some format (the usual case) or for writing data in standard formats so it can be exchanged with other people. Rick White

Rick White writes:
Pickling is fine if all you want to do is create data within Numeric and save it. But it's not useful either for reading data that already exists in some format (the usual case) or for writing data in standard formats so it can be exchanged with other people.
That's right. If you want to read binary formats, for which you know the data layout the numpyio package from Travis Oliphant is very fast and memory efficient, because it builds up the array during the read/write so no copy of the data string needs to be in memory. I see no other way to read other formats generally besides specific wrappers, like the netcdf interface. __Janko Hauser
participants (2)
-
Janko Hauser
-
rlw@stsci.edu