Parsing Search Criteria

Alex Martelli aleax at
Thu Mar 6 23:47:55 CET 2003

'rot-13' wrote:

> I want to parse a criteria string, counting words enclosed in quotes as
> one 'word'. Like this:
>>>> parse_criteria('parrot "I like Python." ternary operator trouble')
> ['I like Python.', 'parrot', 'ternary', 'operator', 'trouble']

I'm confused by the strange ordering you specify in this example.
Without strange reordering, here's how I'd do it:

>>> import tokenize, cStringIO
>>> [ t[1] for t in tokenize.generate_tokens(cStringIO.StringIO('parrot "I 
like Python." ternary operator trouble').readline) if t[0] ]
['parrot', '"I like Python."', 'ternary', 'operator', 'trouble']

Better to wrap this into a function, of course -- for example:

import tokenize, cStringIO
def parse_criteria(criteria):
    rl = cStringIO.StringIO(criteria).readline
    return [ t[1] for t in tokenize.generate_tokens(rl) if t[0] ]


More information about the Python-list mailing list