[python-win32] FIle I/O on Windows XP
cappy2112 at gmail.com
Mon Jun 15 07:39:44 CEST 2009
I was trying to see if I could speed up processing huge files (in the
10's of Gigabytes)
by passing various values to the readline() method of the file object.
No matter what I passed to readline() each call was slower and slower
than passing no argument at all.
I've used a SATA bus analyzer to see what kind of block sizes WIndows
was using for the reads, and typically for non-AHCI reads, the block
sizes were surpsingly small. 8 blocks were typical.
It's somewhat difficult to separate I/O traffic that windows is
routinely doing even when you're not reading a specific file, so
putting the file on a secondary drive will eliminate most of that
Can anyone explain why passing any non-negative values > 0 to
readline() makes the file processing slower instead of faster?
More information about the python-win32