[Twisted-web] Re: [Web-SIG] WSGI woes

[Phillip J. Eby]

At 06:37 PM 9/16/04 -0400, Phillip J. Eby wrote:
>Not entirely, actually.  For my approach to really work, the middleware 
>would have to be guaranteed to return something from read(), as long as 
>the parent's read() returns something.  Otherwise, the resumption would 
>block, unless the middleware were much smarter.  I've got to think about 
>it some more, because right now I'm still not happy with the specifics of 
>any of the proposals for pausing and resuming output.

Aha!  There's the problem.  The 'read()' protocol is what's wrong.  If 
'wsgi.input' were an *iterator* instead of a file-like object, it would be 
fairly straightforward for async servers to implement "would block" reads 
as yielding empty strings.  And, servers could actually support streaming 
input via chunked encoding, because they could just yield blocks once 
they've arrived.

The downside to making 'wsgi.input' an iterator is that you lose control 
over how much data to read at a time: the upstream server or middleware 
determines how much data you get.  But, it's quite possible to make a 
buffering, file-like wrapper over such an iterator, if that's what you 
really need, and your code is synchronous.  (This will slightly increase 
the coding burden for interfacing applications and frameworks that expect 
to have a readable stream for CGI input.)  For asynchronous code, you're 
just going to invoke some sort of callback with each block, and it's the 
callback's job to deal with it.

What does everybody think?  If combined with a "pause iterating me until 
there's input data available" extension API, this would let the input 
stream be non-blocking, and solve the chunked-encoding input issue all in 
one change to the protocol.  Or am I missing something here?