[Python-Dev] RE: [Patches] [Patch #102915] xreadlines : readlines :: xrange : range

[Guido van Rossum]

> [Guido]
> > I only see a factor of 3 between fastest and slowest, and
> > readline is only about 60% slower than readlines_sizehint.
> 
> [Neil]
> > Could it be that your using the CVS version of Python which
> > includes Andrew's cool glibc getline enhancement?
> 
> [Guido]
> > Bingo!
> 
> It's a good thing I haven't yet had time to try any speed tests myself,
> since I don't have a glibc-enabled platform so Guido and I may have been
> tempted to disagree about numbers in public <wink>.
> 
> I checked out the source for glibc's getline.  It's pulling the same trick
> Perl uses, copying directly from the stdio buffer when it can, instead of
> (like Python, and like almost all vendor fgets implementations) doing
> getc-in-a-loop.  The difference is that Perl can't do that without breaking
> into the FILE* representation in platform-dependent ways.  It's a shame that
> almost all vendors missed that fgets was defined as a primitive by the C
> committee precisely so that vendors *could* pull this speed trick under the
> covers.  It's also a shame that Perl did it for them <wink>.

Quite apart from whether we should enable xreadlines(), could you look
into doing a similar thing for MSVC stdio?  For most Unix platforms, a
cop-out answer is "use glibc" -- but for Windows it may pay to do our
own hack.

--Guido van Rossum (home page: http://www.python.org/~guido/)