<div dir="ltr"><br><br><div class="gmail_quote">On Tue, Jul 17, 2012 at 2:59 PM, Serhiy Storchaka <span dir="ltr"><<a href="mailto:storchaka@gmail.com" target="_blank">storchaka@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div class="im">On 17.07.12 06:34, Eli Bendersky wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
The second approach is consistently 10-20% faster than the first one<br>
(depending on input) for trunk Python 3.3<br>
<br>
Is there any reason for this to be so? What does BytesIO give us that<br>
the second approach does not (I tried adding more methods to the patched<br>
RawIOBase to make it more functional, like seekable() and tell(), and it<br>
doesn't affect performance)?<br>
</blockquote>
<br></div>
BytesIO resizes underlying buffer if it overflowed (overallocating 1/8 of size and copying old content to new buffer). Total it makes log[9/8](N) allocations and copy 8*N bytes (for large N). List uses the same strategy, but number of chunks usually significantly less than number of bytes. At the end all this chunks concatenated by join, which calculates sum of chunk lengths and allocate the resulting array with the desired size. That is why append/join is faster than BytesIO in this case.<br>
<br></blockquote><div><br>I've created <a href="http://bugs.python.org/issue15381">http://bugs.python.org/issue15381</a> to track this (optimizing BytesIO).<br> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
There are other note, about ElementTree.tostringlist(). Creating DataStream class in every function call is too expensive, and that is why "monkeypatched" version several times is faster than DataStream version for small data. But for long data it is faster too, because data.append() is on one lookup slower than "monkeypatched" write=data.append.<br>
</blockquote></div><br>I updated tostringlist() to use an outside class. This brings performance back to the old code.<br><br>Eli<br><br><br><br><br><br></div>