[Web-SIG] [extension] x-wsgiorg.flush

Manlio Perillo manlio_perillo at libero.it
Thu Oct 4 18:55:45 CEST 2007


Phillip J. Eby ha scritto:
> At 06:07 PM 10/4/2007 +0200, Manlio Perillo wrote:
>> For nginx mod_wsgi I'm planning to add support to blocking
>> application,executing them in a thread (*but* there will be only one
>> thread per process, and the entire result will be buffered).
>>
>> Threaded execution will be disabled by default, and can be enabled using
>> an option.
>>
>> To add support to asynchronous WSGI application, I will try to implement
>> the pause_output extension and, more important, I will expose the nginx
>> event API to the WSGI application, writing an extension module.
>>
>> The API will be low level, but once this API will be implemented, it
>> should be possibile to implement a common and standardized API, that
>> will works with nginx mod_wsgi and Twisted.
> 
> Will this API support async database connections?  

No.
Async database connections can be implemented using this API.

Using this API we can, as an example, use the asynchronous API already 
implemented by psycopg2 (but not tested, since no one seems to be 
interested):


import psycopg2
import ngx_reactor


def handler(event):
    if cursor.isready():
       resume()

conn = psycopg2.connect(database='test')
curs = conn.cursor()
fileno = curs.fileno()

event = ngx_reactor.create_event(fileno, handler, ...)
ngx_reactor.add_event(event, NGX_READ_EVENT)

resume = environ['wsgi.pause_output']()
curs.execute("SELECT * from sleep(%s, 1)", (delay,), async=1)
yield ''

# Now we have the full response, and we can proceed as in a synchronous 
# application



The real problem here, is the fact that we can not execute new queries 
until the current query terminates, so we need to implement a query queue.

Another big problem is when we want to use a transaction, since we need 
to execute more then one query.


> Async HTTP client 
> operations?  

Again, this is will be a low level API.

However I think that it should be possible to write an "emulation" of a 
Twisted reactor, so we can use the protocols implemented in Twisted (but 
this is a *big* challenge, and I'm not really interested, since if I 
need to use Twisted protocols, then I will use Twisted Web).

> If not, then all it would be good for is waiting for the 
> HTTP input stream.  

The current implementation of nginx mod_wsgi already waits until the 
full request body has been read by Nginx (and the input stream object is 
an instance of cStringIO or File object, depending on the size of the 
request body and the value of the client_body_buffer_size option).

Nginx does not yet implements input filters.

> And if so, then what's the point?


Regards  Manlio Perillo


More information about the Web-SIG mailing list