shlex last token

[Steven Taschuk]

Quoth Peter Maas:
> I am trying to use shlex.shlex and I don't no how to tell that the
> input stream is at EOF (other than using exceptions).

According to the shlex documentation:

    get_token()
        Return a token. If tokens have been stacked using push_token(),
        pop a token off the stack. Otherwise, read one from the input
        stream. If reading encounters an immediate end-of-file, an empty
        string is returned.

This suggests, for example,

    while True:
        token = mylexer.get_token()
        if not token:
            break
        dealwith(token)

or, more tidily,

    for token in iter(mylexer.get_token, ''):
        dealwith(token)

These work for me.  Is your shlex not working this way?

-- 
Steven Taschuk                            staschuk at telusplanet.net
"Our analysis begins with two outrageous benchmarks."
  -- "Implementation strategies for continuations", Clinger et al.