python disk i/o speed

[Pete Shinners]

nnes wrote:
> I will post the source of the programms in a follup post. The
> implementation details are not exactly the same. I spent a couple of
> days on a satisfactory ANSI C version for example and about 20 minutes
> on the python script. :-)

look forward to the code. something like this seems like it might be most 
efficient (warning, parses ok only by my email editor)


def filterfile(inname, outname):
     def intsum(a,b): return int(a)+int(b)
     out = file(outname, 'w')
     for line in file(inname):
         nums = line.split()
         sum = reduce(intsum, nums)
         nums.append(sum)
         newstr = ' '.join(nums)
	out.write(newstr + '\n')


i'm sure the big benefit of something like this over the C version is the 
actual code. easy to write and easy to maintain. i'm sure some perl wizard 
could come and give you the same program in one line of code. the problem 
with that is, it's pretty much a "dead-end" as far as code goes.

i would suspect this runs about the same speed as the java version, but i 
really cannot be sure. this code could also be sped up considerably by 
getting the "int" variable into the local namespace, instead of just 
global. and since you know your lines will have 3 numbers you could 
probably to without the reduce (which should also probably not be global, 
for speed reasons)

def filterfile2(inname, outname):
     localint = int
     out = file(outname, 'w')
     for line in file(inname):
         x, y, z = line.split()
         sum = localint(x) + localint(y) + localint(z)
         newstr = ' '.join((x, y, z, sum))
         out.write(newstr + '\n')

i don't have a 7MB file ready to test these on, but it'd be interesting to 
see the differences.