[python-win32] FIle I/O on Windows XP

Tony Cappellini cappy2112 at gmail.com
Mon Jun 15 07:39:44 CEST 2009


I was trying to see if I could speed up processing huge files (in the
10's of Gigabytes)
by passing various values to the readline() method of the file object.

No matter what I passed to readline() each call was slower and slower
than passing no argument at all.

I've used a SATA bus analyzer to see what kind of block sizes WIndows
was using for the reads, and typically for non-AHCI reads, the block
sizes were surpsingly small. 8 blocks were typical.

It's somewhat difficult to separate I/O traffic that windows is
routinely doing even when you're not reading a specific file, so
putting the file on a secondary drive will eliminate most of that
traffic.

Can anyone explain why passing any non-negative values > 0 to
readline() makes the file processing slower instead of faster?


More information about the python-win32 mailing list