[Web-SIG] Re: Latest WSGI Draft (Phillip J. Eby)

[Phillip J. Eby]

At 11:26 AM 8/24/04 -0500, Ian Bicking wrote:
>Peter Hunt wrote:
>>Is there a "Hello, world!" type of middleware that I could take a look at?
>
>I haven't tested this (but I'll try to tonight), but here's perhaps a more 
>realistic middleware.  This compresses (with gzip) the response from the 
>application (if it is allowed to):
>
>import gzip
>from cStringIO import StringIO
>
>class gzip_middleware(object):
>
>     def __init__(self, application, compress_level=5):
>         self.application = application
>         self.compress_level = compress_level
>
>     def __call__(self, environ, start_response):
>         if 'gzip' not in environ.get('HTTP_ACCEPT'):
>             # nothing for us to do, so this middleware will
>             # be a no-op:
>             return application(environ, start_response)
>         response = GzipResponse(start_response, self.compress_level)
>         app_iter = self.application(environ,
>                                     response.gzip_start_response)
>         response.finish_response(app_iter)
>         return None
>
>class GzipResponse(object):
>
>     def __iter__(self, start_response, compress_level):

I think you meant '__init__' here.


>         self.start_response = start_response
>         self.compress_level = compress_level
>         self.gzip_fileobj = None
>
>     def gzip_start_response(self, status, headers):
>         # This isn't part of the spec yet:
>         if headers.has_key('content-encoding'):
>             # we won't double-encode
>             return self.start_response(status, headers)
>
>         headers['content-encoding'] = 'gzip'
>         raw_writer = self.start_response(status, headers)
>         dummy_fileobj = object()
>         dummy_fileobj.write = raw_writer
>         self.gzip_fileobj = GzipFile('', 'wb', self.compress_level,
>                                      dummy_fileobj)
>         return self.gzip_fileobj.write
>
>     def finish_response(self, app_iter):
>         try:
>             for s in app_iter:
>                 self.gzip_fileobj.write(s)
>         finally:
>             if hasattr(app_iter, 'close'):
>                 app_iter.close()
>             self.gzip_fileobj.close()
>
>
>
>
>Hmm... For a very simple filter, I actually found that surprisingly 
>difficult to write.

Maybe because you used classes unnecessarily?


     class GzipOutput(object):
         pass

     def gzip_middleware(application, compress_level=5):

         def do_gzip(environ, start_response):

             writer = []

             if 'gzip' not in environ.get('HTTP_ACCEPT'):
                 # nothing for us to do, so this middleware will
                 # be a no-op:
                 return application(environ, start_response)

             def gzip_start_response(status, headers):
                 if 'content-encoding' in headers:
                     writer.append(start_response(status,headers))
                 else:
                     headers['content-encoding'] = gzip
                     raw_writer = start_response(status,headers)
                     dummy_fileobj = GzipOutput()
                     dummy_fileobj.write = raw_writer
                     gzip_file = GzipFile('','wb',compress_level,dummy_fileobj)
                     writer.append(gzip_file.write)
                 return writer[0]

             app_iter = application(environ,gzip_start_response)

             if app_iter and writer:
                 try:
                     map(writer[0],app_iter)
                 finally:
                     if hasattr(app_iter,'close'):
                          app_iter.close()
             else:
                 return app_iter

         return do_gzip

Hm.  That's only slightly less complicated.  Still, the only "excise" is 
handling the try/finally for the close -- virtually everything else is 
directly connected to the required functionality.  (By the way, your 
implementation tries to iterate even if the app returns None, and you can't 
set arbitrary attributes on 'object' instances.)

It may be that the PEP should contain a list of suggested utility 
functions, like this one:

     def finish_response(write_func,app_return):
         if app_return:
             try:
                 map(write_func,app_return)
             finally:
                 if hasattr(app_return,'close'):
                     app_return.close()

Such a routine would come in handy for response-munging middleware.


>   And I think it should take advantage of its server's iteration, but 
> currently it only uses the "push" (write function) aspect of the 
> server.  But I'm not sure how exactly I would do that, especially so that 
> the iteration actually had any beneficial properties.

For the given application, it's not important.  Gzipping a server push 
stream probably doesn't make a lot of sense.  :)

If you *really* want to support it, you could do something like:

    def iter_response(transformer, queue, app_return):
        for data in app_return:
            transformer(data)
            if queue:
                yield ''.join(queue)
                queue[:] = []

Where "queue" is a list appended to by the 'transformer'.  For your 
example, you could set it up like this:

     queue = []
     dummy_fileobj.write = queue.append

I'll leave the rest as an exercise for the reader.  :)