writing large files quickly

Grant Edwards grante at visi.com
Fri Jan 27 22:22:18 CET 2006

On 2006-01-27, Erik Andreas Brandstadmoen <erik at brandstadmoen.net> wrote:
> Grant Edwards wrote:
>> Because the filesystem code keeps track of where you are in
>> that 400MB stream, and returns 0x00 anytime you're reading from
>> a "hole".  The "cp" program and the "md5sum" just open the file
>> and start read()ing.  The filesystem code returns 0x00 bytes
>> for all of the read positions that are in the "hole", just like
>> Don said:
> And, this file is of course useless for FS benchmarking, since
> you're barely reading data from disk at all.

Quite right.  Copying such a sparse file is probably only
really testing the write performance of the filesystem
containing the destination file.

> You'll just be testing the FS's handling of sparse files.

Which may be a useful thing to know, but I rather doubt it.

> I suggest you go for one of the suggestions with larger block
> sizes. That's probably your best bet.

Grant Edwards                   grante             Yow!  Now I'm concentrating
                                  at               on a specific tank battle
                               visi.com            toward the end of World
                                                   War II!

More information about the Python-list mailing list