[Chicago] is there really no built-in file/iter split() thing?

skip at pobox.com skip at pobox.com
Sat Dec 1 22:04:02 CET 2007


    me> My candidate:
    ...
    David> scanning the string twice?
    ...
    David> Bug: doesn't work if there are multiple statements on one line.

That's what I get for tossing off something quickly while I'm mostly paying
attention to other things.  Here's a second attempt.  The empty string
output for an empty input is debatable.

Skip

"""
>>> for chunk in chunker(iter(['abc ; def ; ghi;\\n'])):
...   print repr(chunk)
...
'abc '
' def '
' ghi'
'\\n'
>>> for chunk in chunker(iter(['abc;\\n', 'def\\n', 'ghi;\\n'])):
...   print repr(chunk)
...
'abc'
'\\ndef\\nghi'
'\\n'
>>> for chunk in chunker(iter([])):
...   print repr(chunk)
...
''
>>> for chunk in chunker(iter([';', ';;\\n'])):
...   print repr(chunk)
...
''
''
''
'\\n'
"""

def chunker(f, splitter=";"):
    chunks = [""]
    while True:
        for chunk in chunks[:-1]:
            yield chunk
        chunks = chunks[-1:]
        try:
            line = f.next()
        except StopIteration:
            break
        chunks = (chunks[-1] + line).split(splitter)
    for chunk in chunks:
        yield chunk

if __name__ == "__main__":
    import doctest
    doctest.testmod()


More information about the Chicago mailing list