python disk i/o speed

[nnes]

Thanks all of you!

After reading the enlightening comments I came up with a new version.
It looks nicer too :-).

Jeffs code looked like out of a Cookbook: use re for input and % for
output on stdio and you have got almost unlimited flexibility, and if
you understand regular expresions you know exactly what´s going on. It
was slower of course. Now I wish the print format syntax and the re
syntax were somehow unified.

The other approaches were similar to mine using split. Some also using
map and reduce which would be good if I had more than 3 variables. I
will test it later, but I doubt it will improve the speed.

Unfortunately the speedup is not near I would have expected. From the
old 36 seconds down to 33 seconds (using python -O). So I am still
open to suggestions.

cheers all
Nestor

#########################bench2.py:

import sys,xreadlines

def filterfile2(inname, outname):

     inp = file(inname)
     out = file(outname, 'w')

     localint = int

     for line in inp.xreadlines():
         x, y, z = line.split('","')
         x=x[1:]
         z=z[:-2]
         out.write('"%s","%s","%s","%i"\n' % (
          x,y,z,localint(x) + localint(y) + localint(z)))

     out.close()
     inp.close()
   
if len(sys.argv)<3:
    sys.exit("bench input output")

filterfile2(sys.argv[1],sys.argv[2])