Creating huge data in very less time.
python.list at tim.thechases.com
Tue Mar 31 12:33:30 CEST 2009
venutaurus539 at gmail.com wrote:
> On Mar 31, 1:15 pm, Steven D'Aprano
>> The fastest HDDs can reach about 125 MB per second under
>> ideal circumstances, so that will take at least 8 seconds
>> per 1GB file or 8000 seconds in total.
> That time is reasonable.
You did catch the bit about "the *fastest* HDDs" (my emphasis).
Unless you've got some massive RAID or Gig-E/Fiberchanel SAN, you
likely don't have these ideal conditions. Additionally, I've
seen "125MB/sec" as the read speeds -- sustained write speeds are
often lower. Under more real-world testing, you'll likely get
throughput closer to 30-70MB/sec. Call that roughly half the
throughput, and you're up to 16,000 seconds, or about 4.5hr. A
far cry from the "few minutes of time" you first mentioned...
And this doesn't take into consideration the OS overhead of the
filesystem type. Some filesystem types are optimized for large
sequential access, while others work better with smaller files.
You then have things like OS permission overhead, directory-path
overhead, and other disk I/O going on at the same time.
More information about the Python-list