[Web-SIG] Draft language for WSGI to forbid blocking by middleware

[Phillip J. Eby]

I'm proposing the following language to be added to PEP 333, as a 
subsection under "Buffering and Streaming", just before the subsection 
entitled, "The write() callable".  It doesn't address pausing or resuming 
iteration (which I don't have a PEP-able proposal for yet), but it should 
ensure that middleware doesn't introduce any additional blocking issues:

===excerpt start===

Middleware Handling of Block Boundaries
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

In order to better support asynchronous applications and servers,
middleware components **must not** block iteration waiting for
multiple values from an application iterable.  If the middleware
needs to accumulate more data from the application before it can
produce any output, it **must** yield an empty string.

To put this requirement another way, a middleware component **must
yield at least one value** each time its underlying application
yields a value.  If the middleware cannot yield any other value,
it must yield an empty string.

This requirement ensures that asynchronous applications and servers
can conspire to reduce the number of threads that are required
to run a given number of application instances simultaneously.

Note also that this requirement means that middleware **must**
return an iterable as soon as its underlying application returns
an iterable.  It is also forbidden for middleware to use the
``write()`` callable to transmit data that is yielded by an
underlying application.  Middleware may only use their parent
server's ``write()`` callable to transmit data that the
underlying application sent using a middleware-provided ``write()``
callable.

===excerpt end===

In addition to this insertion, I would modify the 'start_response()' 
specification to note that HTTP headers should not be sent until the first 
*non-empty* string is yielded from the iterable.

Comments?