[Web-SIG] wsgi and generators (was Re: WSGI and start_response)

[Graham Dumpleton]

On 10 April 2010 23:04, Chris Dent <chris.dent at gmail.com> wrote:
> On Thu, 8 Apr 2010, P.J. Eby wrote:
>
>> This is also a good time for people to learn that generators are usually a
>> *very bad* way to write WSGI apps - yielding is for server push or sending
>> blocks of large files, not tiny strings.  In general, if you're yielding
>> more than one block, you're almost certainly doing WSGI wrong.  The typical
>> HTML, XML, or JSON output that's 99% of a webapp's requests should be
>> transmitted as a single string, rather than as a series of snippets.
>
> Now the thread that included the quoted bit above has died down a bit, I
> wanted to get back to this. I was surprised when I read this as I found
> it counter intuitive, different to what I'm doing in practical day to
> day WSGI app creation and contrary to what my old school network
> services thinking thinks (start getting stuff queued for the pipe as
> soon as possible).
>
> The apps I'm creating tend to be HTTP APIs that are trying to be RESTful
> and as such they have singular resources I call entities, and aggregates
> of those entities I call collections. The APIs provide access to GETting
> and PUTting entities and GETting collections.
>
> Whenever a GET request is made on an entity or collection, the entity or
> entities involved is serialzed to some string form. When there are many
> entities in a collection, yielding their serialized forms makes semantic
> sense as well as (it appears) resource utiliziation sense.
>
> I realize I'm able to build up a complete string or yield via a
> generator, or a whole bunch of various ways to accomplish things
> (which is part of why I like WSGI: that content is just an iterator,
> that's a good thing) so I'm not looking for a statement of what is or
> isn't possible, but rather opinions. Why is yielding lots of moderately
> sized strings *very bad*? Why is it _not_ very bad (as presumably
> others think)?

Because for a WSGI application, you have absolutely no idea what
actual web server it may run on and what the overheads are of sending
a block of data, let alone many.

In Apache for example, if sent as small blocks, for which a flush has
to occur between each, you have to call into the Apache output filter
bucket chain on every block. This in itself is not an insubstantial
overhead if done many many times. You also have the actual overheads
of writing smalls blocks onto the actual socket.

Let us take an extreme example of a hello world program.

import sys

def application(environ, start_response):
    status = '200 OK'
    output = 'Hello World!'
    response_headers = [('Content-type', 'text/plain'),
                        ('Content-Length', str(len(output)))]
    start_response(status, response_headers)

    return [output]

Say for this I can reliably get:

Requests per second:    2122.56 [#/sec] (mean)

Now change the last line of that hello world program, mirroring a
common mistake you see some make, to:

    return output

so that instead of yielding a single string, yields each character in
the string.

About the best I can achieve now is:

Requests per second:    1973.51 [#/sec] (mean)

This example is only a small string and so only a handful of flushes
had to be done. If you break up a large amount of data into many small
bits, the overheads will obviously become worse. More so if you
actually had Apache output filters installed such as mod_deflate which
actually did work on ever flush. In case above there were no output
filters installed.

So, you may get away with it, but you just have to be a bit careful on
how fine grained you do it. Also, since lot of clients are going to be
slow at reading the response, it is questionable how much it would
help anyway. Delaying and sending as complete response may work just
as well or better. Certainly, if using a front end such as nginx,
returning a complete response will allow the WSGI server to off load
the full response quicker because of the way nginx works as buffer.
Dribbling it in bits just means the backend has to do more work.

Overall I would suggest you form complete responses and focus your
effort instead on better application caching so that you can deliver
responses from a cache and avoid the whole need to generate it in the
first place.

Graham