iostream-like lib?

[Dave Brueck]

On Thu, 15 May 2003, Max Khesin wrote:

> The trouble is that readline() reads more than i have to in the first place,
> even before I call split().
> I did hack it along the lines you suggested with a generator (limiting
> readline() to a number of bytes and accounting for the last character being
> possibly whitespace). I was just wondering if (and why not) there is/is not
> direct support for whitespace-delimited input.

Try this (untested):

def tokenize(fname):
    for line in file(fname):
      for token in line.split():
        yield token

Use it like this:

for token in tokenize('foo.txt'):
  print token

-Dave