[Python-Dev] test_inspect failure

[Tim Peters]

[Tim]
> OK, it's down to this:
>
> >>> f = open('../lib/test/regrtest.py', 'rU')
> >>> len(f.readlines())
> 232
> >>> g = open('../lib/test/regrtest.py')
> >>> len(g.readlines())
> 715
> >>>
>
> IOW, opening in U mode screws up on Windows when universal newlines is
> enabled.
> ...
> For whatever reason, this only delivers the first 232 lines of
> regrtest.py.  I doubt it's a coincidence this is darned near exactly
> the first 8192 bytes of the file.

And it's not:  8192 is the initial buffer size used by file_readlines().
The return value of Py_UniversalNewlineFread() isn't documented, so it's
hard to know what was intended.  It returns the number of bytes in the
buffer.  But file_readlines() believes it returns the number of bytes
actually read from the file.  On Windows the former number is less than the
latter number, by the count of \r\n pairs shrunk to \n.  file_readlines()
thus gets back a number smaller than the 8192 it passed in, and so believes
the file is exhausted by the first read attempt.

I expect the best thing to do is to make ntodo reflect the number of bytes
remaining to be filled in the buffer, rather than the number of bytes
actually read from the file.