4 Sep
2009
4 Sep
'09
5:10 p.m.
Simon Cross
Well, since the source for _read_chunked includes the comment
# XXX This accumulates chunks by repeated string concatenation, # which is not efficient as the number or size of chunks gets big.
you might gain some speed improvement with minimal effort by gathering the read data chunks into a list and then returning "".join(chunks) at the end.
+1 for trying this. Given differences between platforms in realloc() performance, it might be the reason why it goes unnoticed under Linux but degenerates under Windows. As a sidenote, it is interesting that even an stdlib module makes this mistake and acknowledges it without trying to fix it.