seeking thru a file

[Bruno Desthuilliers]

Mag Gam a écrit :
> I have a compressed CSV gziped file. 

Then gunzip it first...

> I was wondering if it is possible
> to seek thru a file
> 
> For example:
> 
> I want to load the first 100 lines into an array. Process the data
> 
> Seek from 101 line to 200 lines. Process the data (remove lines 0 -
> 100) from memory
> 
> Seek 201 to 300 line. Process the data (remove 200-300)) from memory
> 
> etc..etc..

Yes, you can... Now is there any reason you want to micro-manage stuff 
already managed by Python and the file system ?-)

To iterate on a text file's lines, just open the file - the file object 
is it's own iterator, and will (with help from the filesystem AFAIK) 
properly handle cache, buffers and whatnot. The canonical code snippet is:

# iterating over a file line by line without
# loading the while file in memory
f = open("/path/to/some/file.txt")
for line in f:
     do_something_with(line)

f.close()


Unless you have a good reason (micro-managing IO buffers not being one) 
to want to work by 100-lines batch, then this should be just enough.

Now note that Python also have a csv module in the standard lib, which 
will probably (except perhaps in a very few corner cases) be more robust 
and efficient that whatever you'd write.


HTH