Lazy file.readlines()?

Tim Peters tim_one at email.msn.com
Sun Sep 19 18:58:44 EDT 1999


[meow, about his or her qio module at
 http://members.xoom.com/meowing/python/
]
> ...
> Also added some convenience features like user-specified line
> terminators, automatic chopping, and a way to read SMTP-like sockets
> wicked fast.

Yes, qio.readline() is about the most frightening function in the Python
universe now <wink>.

> After adding the requisite bloat, vanilla qio.readline() is still
> about 5-8 times as fast as native Python looping through big text
> files (where stdio is glibc's).

I compiled it for Win32 (btw, "mode_t" was not defined, & there are about a
half dozen warnings about mixing signed and unsigned ints), with
optimization cranked up.  Under Win95 it was between 2-3 times as fast as
1.5.2 Python doing a native readline() loop over an 11Mb text file.  It's
hard to believe that MS's grudging implementation of stdio is that much
better than glibc's, so not sure what to make of this -- possibly that
*nothing* can run really fast under Windows <0.9 wink>.

> ...
> Details at <URL:http://members.xoom.com/meowing/python/>

Nice work!

if-only-windows-had-any-text-files<0.7-wink>-ly y'rs  - tim






More information about the Python-list mailing list