Python 3 read() function
Terry Reedy
tjreedy at udel.edu
Thu Dec 4 13:31:01 EST 2008
Jerry Hill wrote:
> On Thu, Dec 4, 2008 at 11:48 AM, Christian Heimes <lists at cheimes.de> wrote:
>> Cro wrote:
>>> vContent = io.StringIO()
>>> vContent = huge.read() # This line takes hours to process !!!
>> Do you really mean io.StringIO? I guess you want io.BytesIO() ..
>
> I don't think it matters. Here's a quick comparison between 2.5 and
> 3.0 on a relatively small 17 meg file:
>
> C:\>c:\Python30\python -m timeit -n 1
> "open('C:\\work\\temp\\bppd_vsub.csv', 'rb').read()"
> 1 loops, best of 3: 36.8 sec per loop
>
> C:\>c:\Python25\python -m timeit -n 1
> "open('C:\\work\\temp\\bppd_vsub.csv', 'rb').read()"
> 1 loops, best of 3: 33 msec per loop
>
> That's 3 orders of magnitude slower on python3.0!
Timing of os interaction may depend on os. I verified above on WinXp
with 4 meg Pythonxy.chm file. Eye blink versus 3 secs, duplicated. I
think something is wrong that needs fixing in 3.0.1.
http://bugs.python.org/issue4533
tjr
More information about the Python-list
mailing list