Hi. I have started to write an asynchronous WSGI implementation for Twisted Web. The code is available from a Mercurial repository: http://hg.mperillo.ath.cx/twisted/twsgi/ The WSGI application is executed in the main Twisted thread, and the application will be able to directly use Twisted features. The reason I'm doing this is because I have written a WSGI implementation for Nginx: http://hg.mperillo.ath.cx/nginx/ngx_http_wsgi_module and I would like to have a *similar* implementation written in pure Python, for testing purpose. I have some questions: * What's the use of __metaclass__ = type in twisted.web.wsgi ? * Is request.content always buffered (in memory or temporary file)? * In my ngx_http_wsgi_module, when the yielded string can not be sent to the client (because OS buffer is full), I suspend the execution of the application, and resume it when the socket is ready. I want to do the same with Twisted, and the way to go is to be a push producer. However I don't know how to implement and use the IPushProducer interface for _WSGIResponse. By the way: it seems that the WSGI implementation in Twisted Web has some bugs: * close method of the application iterator is not called in case of errors * _sendResponseHeaders does not check if start_response has been called Thanks Manlio
On Apr 9, 2010, at 6:32 AM, Manlio Perillo wrote:
Hi.
I have started to write an asynchronous WSGI implementation for Twisted Web.
The code is available from a Mercurial repository: http://hg.mperillo.ath.cx/twisted/twsgi/
The WSGI application is executed in the main Twisted thread, and the application will be able to directly use Twisted features.
The reason I'm doing this is because I have written a WSGI implementation for Nginx: http://hg.mperillo.ath.cx/nginx/ngx_http_wsgi_module
Maybe put on BitBucket? It'd sure be easier to patch/submit pull requests/file tickets on a public repo.
and I would like to have a *similar* implementation written in pure Python, for testing purpose.
Will these share, or be able to use, the same demo/test code? I'd love to have the option to use Twisted from a WSGI app. Have you resolved the issues you brought up and have been discussing with PJE over on web-sig? Which WSGI are you supporting? This seems like a different thing than the greenlet/mako thing you were discussing over there.
By the way: it seems that the WSGI implementation in Twisted Web has some bugs:
* close method of the application iterator is not called in case of errors
* _sendResponseHeaders does not check if start_response has been called
Before someone else says it... please file bugs (helpfully with test cases) at twistedmatrix.com/trac. S
Steve Steiner (listsin) ha scritto:
On Apr 9, 2010, at 6:32 AM, Manlio Perillo wrote:
Hi.
I have started to write an asynchronous WSGI implementation for Twisted Web.
The code is available from a Mercurial repository: http://hg.mperillo.ath.cx/twisted/twsgi/
The WSGI application is executed in the main Twisted thread, and the application will be able to directly use Twisted features.
The reason I'm doing this is because I have written a WSGI implementation for Nginx: http://hg.mperillo.ath.cx/nginx/ngx_http_wsgi_module
Maybe put on BitBucket? It'd sure be easier to patch/submit pull requests/file tickets on a public repo.
I will think about it. I don't have experience with BitBucket. Is the issue tracker good?
and I would like to have a *similar* implementation written in pure Python, for testing purpose.
Will these share, or be able to use, the same demo/test code?
Yes. The purpose is to have a Twisted and Nginx implementations with the same features.
I'd love to have the option to use Twisted from a WSGI app.
This will be possible with twsgi. However please note that I plan to use it for testing purpose only, and not in production (at least in the near future).
Have you resolved the issues you brought up and have been discussing with PJE over on web-sig? Which WSGI are you supporting?
I will implement WSGI 1.0. Another reason I'm writing twsgi, is that it will be more easy to discuss about WSGI issues, having a pure Python implementation.
This seems like a different thing than the greenlet/mako thing you were discussing over there.
The core is the same. The WSGI implementation *must* support suspend/resume extension. The geenlet middleware will use this extension to implement coroutine support over a simple asynchronous WSGI implementation. By "simple" I mean that greenlets are not used inside WSGI implementation (since greenlets can not be used in my ngx_http_wsgi_module, if I'm not wrong). greenlets are not required, but without coroutines applications will be a mess, due to generator usage.
By the way: it seems that the WSGI implementation in Twisted Web has some bugs:
* close method of the application iterator is not called in case of errors
* _sendResponseHeaders does not check if start_response has been called
Before someone else says it... please file bugs (helpfully with test cases) at twistedmatrix.com/trac.
Of course. But before filling a bug report, I wanted a confirmation. Regards Manlio
On Apr 9, 2010, at 7:20 AM, Manlio Perillo wrote:
Steve Steiner (listsin) ha scritto:
Maybe put on BitBucket? It'd sure be easier to patch/submit pull requests/file tickets on a public repo.
I will think about it. I don't have experience with BitBucket. Is the issue tracker good?
It's more than good enough and free.
and I would like to have a *similar* implementation written in pure Python, for testing purpose.
Will these share, or be able to use, the same demo/test code?
Yes.
The purpose is to have a Twisted and Nginx implementations with the same features.
Cool.
I'd love to have the option to use Twisted from a WSGI app.
This will be possible with twsgi. However please note that I plan to use it for testing purpose only, and not in production (at least in the near future).
The Twisted version for testing only, or both versions for testing only?
Have you resolved the issues you brought up and have been discussing with PJE over on web-sig? Which WSGI are you supporting?
I will implement WSGI 1.0.
Another reason I'm writing twsgi, is that it will be more easy to discuss about WSGI issues, having a pure Python implementation.
This seems like a different thing than the greenlet/mako thing you were discussing over there.
The core is the same. The WSGI implementation *must* support suspend/resume extension.
The geenlet middleware will use this extension to implement coroutine support over a simple asynchronous WSGI implementation.
By "simple" I mean that greenlets are not used inside WSGI implementation (since greenlets can not be used in my ngx_http_wsgi_module, if I'm not wrong).
greenlets are not required, but without coroutines applications will be a mess, due to generator usage.
I guess I'm not real clear on what you're saying here.
By the way: it seems that the WSGI implementation in Twisted Web has some bugs:
* close method of the application iterator is not called in case of errors
* _sendResponseHeaders does not check if start_response has been called
Before someone else says it... please file bugs (helpfully with test cases) at twistedmatrix.com/trac.
Of course. But before filling a bug report, I wanted a confirmation.
If someone can run the unit-test you provide, it's pretty easy to confirm. S
ssteinerX@gmail.com ha scritto:
On Apr 9, 2010, at 7:20 AM, Manlio Perillo wrote:
Steve Steiner (listsin) ha scritto:
Maybe put on BitBucket? It'd sure be easier to patch/submit pull requests/file tickets on a public repo.
I will think about it. I don't have experience with BitBucket. Is the issue tracker good?
It's more than good enough and free.
Well, for free I should be able to get some shared hosting with Mailman and Trac support.
[...]
However please note that I plan to use it for testing purpose only, and not in production (at least in the near future).
The Twisted version for testing only, or both versions for testing only?
Twisted version.
[...]
The geenlet middleware will use this extension to implement coroutine support over a simple asynchronous WSGI implementation.
By "simple" I mean that greenlets are not used inside WSGI implementation (since greenlets can not be used in my ngx_http_wsgi_module, if I'm not wrong).
greenlets are not required, but without coroutines applications will be a mess, due to generator usage.
I guess I'm not real clear on what you're saying here.
This is not simple to explain, for me. It is the reason why I'm trying writing code that will make more easy to document what I'm trying to do ;-)
[...]
Manlio
On 10:32 am, manlio_perillo@libero.it wrote:
Hi.
I have started to write an asynchronous WSGI implementation for Twisted Web.
The code is available from a Mercurial repository: http://hg.mperillo.ath.cx/twisted/twsgi/
The WSGI application is executed in the main Twisted thread, and the application will be able to directly use Twisted features.
The reason I'm doing this is because I have written a WSGI implementation for Nginx: http://hg.mperillo.ath.cx/nginx/ngx_http_wsgi_module
and I would like to have a *similar* implementation written in pure Python, for testing purpose.
I have some questions:
* What's the use of
__metaclass__ = type
in twisted.web.wsgi ?
This makes all classes without a base class defined in the module new- style.
* Is request.content always buffered (in memory or temporary file)?
It's always buffered, in memory if it is small, in a temporary file if it is large (same rules as for a normal request in Twisted Web). Once #288 is resolved, it will probably make sense to stream the request body instead.
* In my ngx_http_wsgi_module, when the yielded string can not be sent to the client (because OS buffer is full), I suspend the execution of the application, and resume it when the socket is ready.
I want to do the same with Twisted, and the way to go is to be a push producer.
However I don't know how to implement and use the IPushProducer interface for _WSGIResponse.
The request object has a register producer method. What else are you having trouble with?
By the way: it seems that the WSGI implementation in Twisted Web has some bugs:
* close method of the application iterator is not called in case of errors
This looks like a bug that should be fixed.
* _sendResponseHeaders does not check if start_response has been called
What are the practical consequences of this? Jean-Paul
exarkun@twistedmatrix.com ha scritto:
On 10:32 am, manlio_perillo@libero.it wrote:
Hi.
I have started to write an asynchronous WSGI implementation for Twisted Web.
[...]
I have some questions:
* What's the use of
__metaclass__ = type
in twisted.web.wsgi ?
This makes all classes without a base class defined in the module new- style.
Ok. Then I assume it is safe to remove it, and derive every class from object.
* Is request.content always buffered (in memory or temporary file)?
It's always buffered, in memory if it is small, in a temporary file if it is large (same rules as for a normal request in Twisted Web).
Good. It is the same with Nginx.
[...]
* In my ngx_http_wsgi_module, when the yielded string can not be sent to the client (because OS buffer is full), I suspend the execution of the application, and resume it when the socket is ready.
I want to do the same with Twisted, and the way to go is to be a push producer.
However I don't know how to implement and use the IPushProducer interface for _WSGIResponse.
The request object has a register producer method. What else are you having trouble with?
I was just not sure if I just have to call registerProducer. I have implemented it: http://hg.mperillo.ath.cx/twisted/twsgi/rev/22738dce8721 Now the WSGI implementation is complete. Still not sure if I should call unregisterProducer in the case notifyFinish callback is called. It seems that it is not required.
By the way: it seems that the WSGI implementation in Twisted Web has some bugs:
* close method of the application iterator is not called in case of errors
This looks like a bug that should be fixed.
I will open a ticket, unless someone else already opened it.
* _sendResponseHeaders does not check if start_response has been called
What are the practical consequences of this?
That in case WSGI application does not call start_response, _sendResponseHeaders will fail with an AttributeError, since self.status is not available. Thanks Manlio
Manlio Perillo ha scritto:
Hi.
I have started to write an asynchronous WSGI implementation for Twisted Web.
The code is available from a Mercurial repository: http://hg.mperillo.ath.cx/twisted/twsgi/
[...] I have some questions:
Now that base features are implemented I would like to ask some more questions. * I choose, as project name twsgi. However I found that this name is also used in a Twisted sandbox: http://github.com/Deepwalker/deepwalker_sandbox/blob/3786de5d77bd68872306e5f... Should I change the name? * Suppose that a client connected to WSGI serve is very slow. In this case, once the server socket buffer is full, WSGI producer will not be resumed. Is this case handled by HTTP server? With Nginx implementation I set a timer, whose value is specified in send_timeout configuration parameter http://wiki.nginx.org/NginxHttpCoreModule#send_timeout It timer expires, I return "408 Request Time-out" From source code, I see that timeout in twisted.web, is specified in request.site.timeOut * Do you have interest of include this implementation in twisted.web?
[...]
Thanks Manlio
participants (4)
-
exarkun＠twistedmatrix.com -
Manlio Perillo -
ssteinerX＠gmail.com -
Steve Steiner (listsin)