seeking thru a file
bruno.42.desthuilliers at websiteburo.invalid
Fri Jun 26 14:34:52 CEST 2009
Mag Gam a écrit :
> I have a compressed CSV gziped file.
Then gunzip it first...
> I was wondering if it is possible
> to seek thru a file
> For example:
> I want to load the first 100 lines into an array. Process the data
> Seek from 101 line to 200 lines. Process the data (remove lines 0 -
> 100) from memory
> Seek 201 to 300 line. Process the data (remove 200-300)) from memory
Yes, you can... Now is there any reason you want to micro-manage stuff
already managed by Python and the file system ?-)
To iterate on a text file's lines, just open the file - the file object
is it's own iterator, and will (with help from the filesystem AFAIK)
properly handle cache, buffers and whatnot. The canonical code snippet is:
# iterating over a file line by line without
# loading the while file in memory
f = open("/path/to/some/file.txt")
for line in f:
Unless you have a good reason (micro-managing IO buffers not being one)
to want to work by 100-lines batch, then this should be just enough.
Now note that Python also have a csv module in the standard lib, which
will probably (except perhaps in a very few corner cases) be more robust
and efficient that whatever you'd write.
More information about the Python-list