[Python-Dev] How io.IOBase.readline() should behave when used on non-blocking obj and no data available?

[Antoine Pitrou]

On Thu, 16 Oct 2014 03:54:32 +0300
Paul Sokolovsky <pmiscml at gmail.com> wrote:
> Hello,
> 
> io.RawIOBase.read() is well specified for behavior in case it
> immediately gets a would-block condition: "If the object is in
> non-blocking mode and no bytes are available, None is returned."
> (https://docs.python.org/3/library/io.html#io.RawIOBase.read).
> 
> However, nothing is said about such condition for io.IOBase.readline(),
> which is mixin method in a base class, default implementation of which
> thus would use io.RawIOBase.read(). Looking at 3.4.0 source, iobase.c:
> iobase_readline() has:
> 
>         b = _PyObject_CallMethodId(self, &PyId_read, "n", nreadahead);
> [...]
>         if (!PyBytes_Check(b)) {
>             PyErr_Format(PyExc_IOError,
>                          "read() should have returned a bytes object, "
>                          "not '%.200s'", Py_TYPE(b)->tp_name);
> 
> I.e. it's not even ready to receive legitimate return value of None
> from read(). I didn't try to write a testcase though, so may be missing
> something.
> 
> So, how readline() should behave in this case, and can that be
> specified in the Library Reference?

Well, the problem is that it's not obvious how to implement such methods
in a non-blocking context.

Let's says some data is received but there isn't a complete line.
Should readline() return just that data (an incomplete line)? That
breaks the API's contract. Should readline() buffer the incomplete line
and keep it for the next readline() call? But then the internal buffer
becomes unbounded: perhaps there is no new line in the next 4GB of
incoming data...

And besides, raw I/O objects *shouldn't* have an internal buffer. That's
the role of the buffered I/O layer.

Regards

Antoine.