[Web-SIG] Proposed specification: waiting for file descriptor events

[Manlio Perillo]

Christopher Stawarz ha scritto:
> This is the third draft of my proposed extensions for better supporting 
> WSGI apps on asynchronous servers.  The major changes since the last 
> draft are as follows:
> 

First of all, thanks for your effort.

> * The title and abstract now accurately reflect the scope of the proposal.
>   In addition, the extensions are now in the namespace "x-wsgiorg.fdevent"
>   (instead of "x-wsgiorg.async").
> 
> * The proposal for an alternative, non-blocking input stream has been
>   dropped, since I don't see a way to add one that wouldn't break 
> middleware.

Well, IMHO the "general" solution here is to use greenlets.

>   Instead, the spec recommends that async servers pre-read the request body
>   before invoking the app (either by default or as a configurable option).
> 

This is the best solution most of the time (but not for all of the 
time), especially if the "server" can do some "pre-parsing" of 
multipart/form-data request body.

In fact I plan to write a custom function (in C for Nginx) that will 
"reduce", as an example:

    Content-Type: multipart/form-data; boundary=AaB03x

    --AaB03x
    Content-Disposition: form-data; name="submit-name"

    Larry
    --AaB03x
    Content-Disposition: form-data; name="files"; filename="file1.txt"
    Content-Type: text/plain

    ... contents of file1.txt ...
    --AaB03x--

to (not properly escaped):

Content-Type: application/x-www-form-urlencoded

submit-name=Larry&files.filename=file1.txt&files.ctype=text/plain&files.path=xxx


and the contents of file1.txt will be saved to a temporary file 'xxx'.


> 
> Once again, I'd appreciate your comments.
>


I have some comments:

1) Why not add a more generic poll like interface?

    Moreover IMHO storing a timeout variable in the environ to check if
    the previous call timedout, is not the best solution.

    In my implementation I return a function, but with generators in
    Python 2.5 this can be done in a better way.

2) In Nginx it is not possible to simply handle "plain" file
    descriptors, since these are wrapped in a connection structure.

    This is the reason why I had to add a connection_wrapper function in
    my WSGI module for Nginx.

3) If you read an example that implements a database connection pool:
http://hg.mperillo.ath.cx/nginx/mod_wsgi/file/tip/examples/nginx-postgres-async.py

    you can see that there is a problem.

    In fact the pool is not very flexible; the application can not handle
    more than POOL_SIZE concurrent requests.

    However it is possible to just have a new request to wait until a
    previous connection is free (or a timeout occurs).

    I have attached an example (it is not in the repository since there
    are some problems).

    The examples use a new extension:

      - ctx = environ['ngx.request_context']()
      - ctx.resume()

    ctx.resume() "asynchronously" resumes the given request
    (it will be resumed as soon as control returns to Nginx, when the
     application yields something).


    Note that the problem of resuming another request is easily solved
    with greenlets, without the need to new extensions
    (this is one of the reason why I like greenlets).


 > [...]



Regards  Manlio Perillo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: nginx-postgres-async-2.py
Type: text/x-python
Size: 4155 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/web-sig/attachments/20080521/4e5ccc3b/attachment.py>