<div class="gmail_quote">On Tue, Jan 27, 2009 at 6:15 PM, Antoine Pitrou <span dir="ltr"><<a href="mailto:solipsis@pitrou.net">solipsis@pitrou.net</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
It's some arbitrary text composed of 95% ASCII characters and 5% non-ASCII. On<br>
this specific example, utf8 decodes at around 250 MB/s, latin1 at almost 1 GB/s<br>
(on the same machine on which I ran the benchmarks).<br>
</blockquote></div><br>For the "10MB whole contents at once" test, we then have:<br>(assuming the code does no pipelining of disk I/O with decoding)<br>
<br>10MB / 980MB/s to read from disk = 10 ms<br>10MB / 250MB/s to decode to utf8 = 40 ms<br>10MB / (10ms + 40ms) = 200 MB/s <br><br>In practice, your results shows around 90 MB/s. That's at least vaguely in the same ballpark.<br>
<blockquote style="margin: 1.5em 0pt;">--<br>
Daniel Stutzbach, Ph.D.<br>
President, <a href="http://stutzbachenterprises.com">Stutzbach Enterprises, LLC</a>
</blockquote>