Question about file objects...

[r0g]

J wrote:
> Something that came up in class...
> 
> when you are pulling data from a file using f.next(), the file is read
> one line at a time.
> 
> What was explained to us is that Python iterates the file based on a
> carriage return as the delimiter.
> But what if you have a file that has one line of text, but that one
> line has 16,000 items that are comma delimited?
> 
> Is there a way to read the file, one item at a time, delimited by
> commas WITHOUT having to read all 16,000 items from that one line,
> then split them out into a list or dictionary??
> 
> Cheers
> Jeff
> 


Generators are good way of dealing with that sort of thing...

http://dalkescientific.com/writings/NBN/generators.html

Have the generator read in large chunks from file in binary mode then
use string searching/splitting to dole out records one at a time,
topping up the cache when needed.

Roger.