[Numpy-discussion] R: fast numpy.fromfile skipping data chunks
Charles R Harris
charlesr.harris at gmail.com
Wed Mar 13 19:40:00 EDT 2013
On Wed, Mar 13, 2013 at 9:54 AM, Andrea Cimatoribus <
Andrea.Cimatoribus at nioz.nl> wrote:
> Thanks a lot for the feedback, I'll try to modify my function to overcome
> this issue.
> Since I'm in the process of buying new hardware too, a slight OT (but
> definitely related).
> Does an ssd provide substantial improvement in these cases?
It should. Seek time on an ssd is quite low, and readout is fast. Skipping
over items will probably not be as fast as a sequential read but I expect
it will be substantially faster than a disk. Nathaniel's loop idea will
probably work faster also. The sequential readout rate of a modern ssd will
be about 500 MB/sec, so you can probably just divide that into your file
size to get an estimate of the time needed.
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the NumPy-Discussion