Reading a large csv file

[Peter Otten]

Mag Gam wrote:

> Yes, the system has 64Gig of physical memory.
> 
> 
> What I meant was, is it possible to load to a hdf5 dataformat
> (basically NumPy array) without reading the entire file at first? I
> would like to splay to disk beforehand so it would be a bit faster
> instead of having 2 copies in memory.

It is certainly possible to read a csv file one line at a time. What exactly 
are you doing to convert it into hdf5? 

Showing some of your code might help.

Peter