Cleaner idiom for text processing?

[Jeff Epler]

I'd move the logic that turns the file into the form you want to
process, under the assumption that you'll use this code from multiple
places.  
    def process_tokens(f):
        for line in infile:
            tokens = line.split()
            d = {}
            for i in range(0, len(tokens), 2): d[tokens[i]] = tokens[i+1]
            yield d
Then,
    for d in process_tokens(infile):
        do_something_with_values(d['foo'], d['bar'])

If the specific keys you want from each line are constant for the loop,
have process_tokens yield those items in sequence:
    def process_tokens2(f, keys):
        for line in infile:
            tokens = line.split()
            d = {}
            for i in range(0, len(tokens), 2): d[tokens[i]] = tokens[i+1]
            yield [d[k] for k in keys]

    for foo, bar in process_tokens(infile, "foo", "bar"):
        do_something_with_values(foo, bar)

Jeff