[Web-SIG] Prototype of wsgi.input.readline().

[Graham Dumpleton]

On 31/01/2008, Chris McDonough <chrism at plope.com> wrote:
> Graham Dumpleton wrote:
> > As I think we all know, no one implements readline() for wsgi.input as
> > defined in the WSGI specification. The reason for this is that stuff
> > like cgi.FieldStorage would refuse to work and would just generate an
> > exception. This is because cgi.FieldStorage expects to pass an
> > argument to readline().
>
> I haven't been keeping up on the issues this has caused wrt WSGI, but note that
> the reason that cgi.FieldStorage passes a size argument to readline is in order
> to prevent memory exhaustion when reading files that don't have any linebreaks
> (denial of service).  See http://bugs.python.org/issue1112549 .

The interesting comment in that bug is:

"""The input data
is not required by the RFC 822/1521/1522/1867
specifications to contain any newline characters."""

If that can occur, then a WSGI adapter which didn't simulate EOF would
fail in that the read would block and never return. All the more
reason that simulating EOF needs to be mandatory.

> > So, although this is linked in the issues list for possible amendments
> > to WSGI specification, there hasn't that I recall been a discussion on
> > how readline() would be defined in any amendment or future version.
> >
> > In particular, would the specification be changed to either:
> >
> > 1. readline(size) where size argument is mandatory, or:
> >
> > 2. readline(size=-1) where size argument is optional.
> >
> > If the size argument is made mandatory, then it would parallel how
> > read() function is defined, but this in itself would mean
> > cgi.FieldStorage would break.
> >
> > This is because cgi.FieldStorage actually calls readline() with no
> > argument as well as an argument in different places in the code.
>
> cgi.FieldStorage doesn't call readline() without an argument.
> cgi.parse_multipart does, but this function is not used by cgi.FieldStorage.  I
> don't know if this changes anything.

Not really, I should have said 'cgi' module as a whole rather than
specifically cgi.FieldStorage. Given that people might be using
cgi.parse_multipart in standard CGI, there would probably still be an
expectation that it worked for WSGI. We can't really say that you can
use cgi.FieldStorage but not cgi.parse_multipart. People will just
expect all the normal tools people would use for this to work.

Graham