shlex last token

Steven Taschuk staschuk at
Sat Jun 14 04:00:49 CEST 2003

Quoth Peter Maas:
> I am trying to use shlex.shlex and I don't no how to tell that the
> input stream is at EOF (other than using exceptions).

According to the shlex documentation:

        Return a token. If tokens have been stacked using push_token(),
        pop a token off the stack. Otherwise, read one from the input
        stream. If reading encounters an immediate end-of-file, an empty
        string is returned.

This suggests, for example,

    while True:
        token = mylexer.get_token()
        if not token:

or, more tidily,

    for token in iter(mylexer.get_token, ''):

These work for me.  Is your shlex not working this way?

Steven Taschuk                            staschuk at
"Our analysis begins with two outrageous benchmarks."
  -- "Implementation strategies for continuations", Clinger et al.

More information about the Python-list mailing list