[Web-SIG] ngx.poll extension (was Re: Are you going to convert Pylons code into Python 3000?)

[Manlio Perillo]

Graham Dumpleton ha scritto:
> On 07/03/2008, Manlio Perillo <manlio_perillo at libero.it> wrote:
>>  Moreover with an asynchronous gateway it is possible to implement a
>>  "middleware" that can execute an application inside a thread.
>>
>>  This is possible by creating a pipe, starting a new thread, having the
>>  main thread polling the pipe, and having the thread write some data in
>>  the pipe to "wake" the main thread when finish its job.
>>
>>  I'm going to write a sample implementation when I find some time.
>>
>>  Yes, we need to use a thread, but this can be done in pure Python code
>>  only (altought I'm not sure if this can have side effects on Nginx).
> 
> So you do understand this technique of using a socketpair() pipe as a
> way of communicating between code which is thread safe and other code
> which is potentially non thread safe. 

Right.

> This makes moot your prior point
> that they (threads) are not supported by the server and thus you want
> to avoid using them.
> 

Not really true ;-).

Threads are still not supported by Nginx.
This means that using threads in an application embedded in Nginx can 
cause who knows what problems (ok, probabily it will *not* cause any 
problems).

Moreover, I'm not sure that such a *middleware* will be a full WSGI 1.0 
conforming middleware.

> In other words, as I have pointed out previously, in practice it would
> be possible to implement a thread pool mechanism on top of nginx such
> that you could avoid this whole problem of the asynchronous model at
> the WSGI level.
> 

No, this does not solves the problem.

The number of threads I can create is limited, so I can serve only a 
limited number of concurrent requests.

The asynchronous solution is more optimized.

> I still don't understand why you are so resistant to going this path
> given that for Python web applications, the event driven model doesn't
> necessarily provide any benefits when one looks at the bigger picture
> and perhaps just makes it harder to implement application code.
> 

The event drive model *does* provide benefits for my problem.

And I'm not looking at the bigger picture here.
I'm looking at a HTTP resource that needs to execute an HTTP request to 
an external web application.

> If you want to pursue an even driven model because you find it an
> interesting area to work in then fine, but you shouldn't expect
> everyone else to try and accommodate that way of thinking when people
> are happy with the alternative.
> 

The problem here is that I'm just pointing out that the *current* WSGI 
1.0 *supports* asynchronous applications.

Until now nobody else have implemented asynchronous applications on top 
of WSGI, and so Philip J. Eby have decided to getting rid of the 
asynchronous support, for the sake of having a simplified implementation.

I'm only saying: "hey, wait. Actually WSGI 1.0 *can* really be used for 
writing asynchronous applications, here is a *working* and not pure 
academic example".



> Graham
> 


Manlio Perillo