Can you do it faster? (parsing text file)

Terry Reedy tjreedy at
Tue Jan 28 21:19:18 CET 2003

"Marcus Stojek" <stojek at> wrote in message
news:3e3650fc.2103718 at
> Hi,
> I have to parse large txt-files (15Mb and more, 200000 lines).
> The file consits of comma separated keywords and alphanumerical
> Each command line starts with some or no blanks, a '*' and the
> followed by comma separated paramters

I would separate the problem into two parts: 1) divide the input into
blocks (each headed by a command line); 2) turn each block into an
entry for the output list.  I would write the block generator as
that -- a generator, which lets you yield at precisely the right

def blocks(infile):
  block = []
  for line in file(infile)
    if iscommand(line):
       if block: yield block
       block = [line]
    elif isnotcomment(line):

Terry J. Reedy

More information about the Python-list mailing list