.readline() - VERY SLOW compared to PERL

Fredrik Lundh fredrik at effbot.org
Sat Nov 18 20:28:28 CET 2000


Andrew wrote:
> http://www.deja.com/[ST_rn=ps]/getdoc.xp?AN=527209193&fmt=text
> > Under Win95 it was between 2-3 times as fast as 1.5.2 Python
> > doing a native readline() loop over an 11Mb text file.

note that readlines is 2-3 times as fast as a naive readline
loop on that platform...

> However, I didn't test 'readline' but rather 'readlines' since I
> could assume my input file streams all supported that method.
>
> There was almost no difference between the builtin file readlines
> and the qio.readlines when I used a sizehint of 1000000 to read a
> file of about 230 MB.  In fact, they were so close in timings that
> I had to double check that I was really calling the right functions.
> Looking at the internals of the two, that's because they are
> implemented essentially the same way.

this matches my experiences from the fastio project -- there
was almost no difference between readlines and any other method
I tried (memory mapping, low-level posix I/O, or windows-specific
I/O code)...

</F>





More information about the Python-list mailing list