[Python-Dev] Python 3.0.1 (io-in-c)
Brett Cannon
brett at python.org
Wed Jan 28 00:02:13 CET 2009
On Tue, Jan 27, 2009 at 14:44, Antoine Pitrou <solipsis at pitrou.net> wrote:
> Raymond Hettinger <python <at> rcn.com> writes:
>>
>> What is involved in finishing io-in-c?
>
> Off the top of my head:
> - fix the _ssl bug which prevents some tests from passing (issue #4967)
> - clean up io.py (and decide what to do with the remaining Python code:
> basically, the parts of StringIO which are implemented in Python)
The other VMs might appreciate the code being available and used if
_io is not available for import. If you need help on how to have the
tests run twice, once on the Python code and again on the C code, you
can look at test_heapq and test_warnings for approaches.
> - of course, test in various situations, review the code, suggest possible
> improvements...
>
> Now here are some performance figures. Text I/O is done in utf-8 with universal
> newlines enabled:
>
That is impressive! Congrats to you and (I think) Amaury for all the
hard work you guys have put in.
-Brett
>
> === I/O in C ===
>
> ** Binary input **
>
> [ 400KB ] read one unit at a time... 1.64 MB/s
> [ 400KB ] read 20 units at a time... 27.2 MB/s
> [ 400KB ] read 4096 units at a time... 845 MB/s
>
> [ 20KB ] read whole contents at once... 924 MB/s
> [ 400KB ] read whole contents at once... 883 MB/s
> [ 10MB ] read whole contents at once... 980 MB/s
>
> [ 400KB ] seek forward one unit at a time... 0.528 MB/s
> [ 400KB ] seek forward 1000 units at a time... 516 MB/s
> [ 400KB ] alternate read & seek one unit... 1.33 MB/s
> [ 400KB ] alternate read & seek 1000 units... 490 MB/s
>
> ** Text input **
>
> [ 400KB ] read one unit at a time... 2.28 MB/s
> [ 400KB ] read 20 units at a time... 29.2 MB/s
> [ 400KB ] read one line at a time... 71.7 MB/s
> [ 400KB ] read 4096 units at a time... 97.4 MB/s
>
> [ 20KB ] read whole contents at once... 108 MB/s
> [ 400KB ] read whole contents at once... 112 MB/s
> [ 10MB ] read whole contents at once... 89.7 MB/s
>
> [ 400KB ] seek forward one unit at a time... 0.0904 MB/s
> [ 400KB ] seek forward 1000 units at a time... 87.4 MB/s
>
> ** Binary append **
>
> [ 20KB ] write one unit at a time... 0.668 MB/s
> [ 400KB ] write 20 units at a time... 12.2 MB/s
> [ 400KB ] write 4096 units at a time... 722 MB/s
> [ 10MB ] write 1e6 units at a time... 1529 MB/s
>
> ** Text append **
>
> [ 20KB ] write one unit at a time... 0.983 MB/s
> [ 400KB ] write 20 units at a time... 16 MB/s
> [ 400KB ] write 4096 units at a time... 236 MB/s
> [ 10MB ] write 1e6 units at a time... 261 MB/s
>
> ** Binary overwrite **
>
> [ 20KB ] modify one unit at a time... 0.677 MB/s
> [ 400KB ] modify 20 units at a time... 12.1 MB/s
> [ 400KB ] modify 4096 units at a time... 382 MB/s
>
> [ 400KB ] alternate write & seek one unit... 0.212 MB/s
> [ 400KB ] alternate write & seek 1000 units... 173 MB/s
> [ 400KB ] alternate read & write one unit... 0.827 MB/s
> [ 400KB ] alternate read & write 1000 units... 276 MB/s
>
> ** Text overwrite **
>
> [ 20KB ] modify one unit at a time... 0.296 MB/s
> [ 400KB ] modify 20 units at a time... 5.69 MB/s
> [ 400KB ] modify 4096 units at a time... 151 MB/s
>
>
> === I/O in Python (branches/py3k) ===
>
> ** Binary input **
>
> [ 400KB ] read one unit at a time... 0.174 MB/s
> [ 400KB ] read 20 units at a time... 3.44 MB/s
> [ 400KB ] read 4096 units at a time... 246 MB/s
>
> [ 20KB ] read whole contents at once... 443 MB/s
> [ 400KB ] read whole contents at once... 216 MB/s
> [ 10MB ] read whole contents at once... 274 MB/s
>
> [ 400KB ] seek forward one unit at a time... 0.188 MB/s
> [ 400KB ] seek forward 1000 units at a time... 182 MB/s
> [ 400KB ] alternate read & seek one unit... 0.0821 MB/s
> [ 400KB ] alternate read & seek 1000 units... 81.2 MB/s
>
> ** Text input **
>
> [ 400KB ] read one unit at a time... 0.218 MB/s
> [ 400KB ] read 20 units at a time... 3.8 MB/s
> [ 400KB ] read one line at a time... 3.69 MB/s
> [ 400KB ] read 4096 units at a time... 34.9 MB/s
>
> [ 20KB ] read whole contents at once... 70.5 MB/s
> [ 400KB ] read whole contents at once... 81 MB/s
> [ 10MB ] read whole contents at once... 68.7 MB/s
>
> [ 400KB ] seek forward one unit at a time... 0.0709 MB/s
> [ 400KB ] seek forward 1000 units at a time... 67.3 MB/s
>
> ** Binary append **
>
> [ 20KB ] write one unit at a time... 0.15 MB/s
> [ 400KB ] write 20 units at a time... 2.88 MB/s
> [ 400KB ] write 4096 units at a time... 346 MB/s
> [ 10MB ] write 1e6 units at a time... 728 MB/s
>
> ** Text append **
>
> [ 20KB ] write one unit at a time... 0.0814 MB/s
> [ 400KB ] write 20 units at a time... 1.51 MB/s
> [ 400KB ] write 4096 units at a time... 118 MB/s
> [ 10MB ] write 1e6 units at a time... 218 MB/s
>
> ** Binary overwrite **
>
> [ 20KB ] modify one unit at a time... 0.123 MB/s
> [ 400KB ] modify 20 units at a time... 2.34 MB/s
> [ 400KB ] modify 4096 units at a time... 213 MB/s
>
> [ 400KB ] alternate write & seek one unit... 0.0816 MB/s
> [ 400KB ] alternate write & seek 1000 units... 71.4 MB/s
> [ 400KB ] alternate read & write one unit... 0.0448 MB/s
> [ 400KB ] alternate read & write 1000 units... 41.1 MB/s
>
> ** Text overwrite **
>
> [ 20KB ] modify one unit at a time... 0.0723 MB/s
> [ 400KB ] modify 20 units at a time... 1.36 MB/s
> [ 400KB ] modify 4096 units at a time... 88.3 MB/s
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
More information about the Python-Dev
mailing list