Itamar and I had dinner today and talked about new-web a bit. Here is what was discussed: 1) We need to keep compatibility with old Nevow resources 2) We ought to keep compatibility with old twisted.web resources 3) The new request API isn't the same as old request API. 4) new-web depends on Twisted 2.0. Nevow will be depending on new-web, so the next Nevow will depend on twisted 2.0. Thus it cannot be released until Twisted 2.0. According to itamar the goal for next Twisted release is around halloween, although I hadn't seen that mentioned anywhere before. ;) 5) Rumors are Anthony is writing a non-sucky HTTP client. (is this true?). J. Simms (slyphon) mentioned something about writing one to me a few weeks ago but I didn't hear anything after. This should be coordinated so the API for the client and server is as similar as possible. Conclusions for action: 1) new-web will be called twisted.web2. I will put what I've got into svn trunk as twisted.web2 tomorrow, assuming no objections. This will allow easy backwards compatibility at the cost of a 2. 2) Nevow's resource traversal API must be moved to twisted.web2. This will probably have to be done by branching Nevow to remove those bits. Therefore, the resource traversal part of nevow should be mostly-frozen while this happens. 3) For distribution: twisted.web2 depends on Nevow (for static directory listings, and error pages), so will be distributed with it. Nevow recommends Twisted but does not require it for simple cases (e.g. flattening to a string from a CGI script), so there can also be a limited-functionality distribution of standalone Nevow. Current status of new-web: the low-level HTTP protocol and header parsing is in passable shape. The high-level Resource API needs to be copied over from Nevow (again), because Nevow's API changed. Backwards compatibility adapters for the t.w.Request/nevow.inevow.IRequest and (possibly) nevow.inevow.IResource should be added. I am not able to do everything myself. I will need help, especially from dp would be helpful for the Nevow transition, to make this happen. If we do it right, twisted.web2 might even be ready in time for the release of Twisted 2.0. James
James Y Knight <foom@fuhm.net> writes: ...
3) For distribution: twisted.web2 depends on Nevow (for static directory listings, and error pages), so will be distributed with it. Nevow recommends Twisted but does not require it for simple cases (e.g. flattening to a string from a CGI script), so there can also be a limited-functionality distribution of standalone Nevow.
What about the wsgi spec ? http://www.python.org/peps/pep-0333.html It could be great if the server could not depend on any framework (i mean nevow). -- William - http://flibuste.net
On Sep 15, 2004, at 12:26 PM, William Dode wrote:
James Y Knight <foom@fuhm.net> writes:
3) For distribution: twisted.web2 depends on Nevow (for static directory listings, and error pages), so will be distributed with it. Nevow recommends Twisted but does not require it for simple cases (e.g. flattening to a string from a CGI script), so there can also be a limited-functionality distribution of standalone Nevow.
What about the wsgi spec ? http://www.python.org/peps/pep-0333.html
The WSGI spec is near irrelevant for a twisted web application. There are two components to look at here: 1) Can we support running WSGI applications? WSGI specifies blocking behavior and thus cannot be used in twisted as the primary, or even a recommended, interface. It is possible to write a WSGI application runner for twistedweb, but it would have to either spawn a thread or process for each simultaneous WSGI request. Twisted's whole architecture is geared towards not having to do that, thus, an API that requires it is decidedly second-class and something that isn't very high on my list of things I care about. 2) Can we run *as* a WSGI application? No, not really. WSGI has 4 different modes a client might be run in: 1) threads in the same process for each simultaneous request, 2) a process for each simultaneous request 3) multiple processes and a thread in one of those processes for each simultaneous request (kind of like 1&2 combined). 4) Spawn a new process for each request and the process ends when the request is done. Twisted web prefers to run all requests in a single thread. Therefore, none of the alternatives are very well matches. It could probably work in mode 2 and 4, although not ideally, but not in mode 1 and 3, because twisted is generally not threadsafe, so running essentially two copies of twisted in one process will not work. I wrote preliminary support for running twisted as a CGI (essentially equivilent to wsgi mode 4) except more generally useful. Clearly this is not ideal but I think it's useful to have a simple way to integrate some twisted stuff into a hostile environment. We could make a WSGI application that just send the requests over a socket to twisted web running in a separate process, but there's not really any reason to IMO. HTTP is already a good protocol for talking over sockets. Use apache mod_proxy to send requests to the twisted.web process. There's also the SCGI socket protocol which looks reasonable and we might want to support.
It could be great if the server could not depend on any framework (i mean nevow).
It will use nevow to render the pages the webserver generates, such as errors and directory listings. This does not mean you need to use nevow for your own pages. You can use whatever templating language you want (or none at all) to render your pages, assuming you can hook it up to twisted web somehow. If you want to make a custom hacked-up copy of twisted.web without Nevow included, it will work, other than default server-generated pages. However, I don't see any reason to not include it by default. James
On Sep 15, 2004, at 12:26 PM, William Dode wrote:
James Y Knight <foom@fuhm.net> writes:
...
3) For distribution: twisted.web2 depends on Nevow (for static directory listings, and error pages), so will be distributed with it. Nevow recommends Twisted but does not require it for simple cases (e.g. flattening to a string from a CGI script), so there can also be a limited-functionality distribution of standalone Nevow.
What about the wsgi spec ? http://www.python.org/peps/pep-0333.html
It could be great if the server could not depend on any framework (i mean nevow).
The server would depend on Nevow in order to generate the HTML for various automatically generated pages, such as directory listings. I suggested to James that having a standalone twisted.web server that didn't depend on Nevow and wasn't capable of generating any pages itself would be possible, but James didn't think it was worth the effort. The WSGI spec is unsuitable for use with asynchronous servers and applications. Basically, once the application callable returns, the server (or "gateway" as wsgi calls it) must consider the page finished rendering. I have implemented a simple WSGI application callable in Nevow svn head, which will take a Page and synchronously render it. However, unless there is a way for the application to continue rendering after it has returned, I don't think this will be much use beyond just simple CGI style apps. I haven't mentioned anything about this to any WSGI list yet because I don't have a solution or a proposal. My initial thought is to have the start_response callable return a tuple of (write, finish) callables, instead of just write as it does now. I'm not yet sure this will actually work, though, so I'll have to give it further thought. dp
Donovan Preston <dp@ulaluma.com> writes:
On Sep 15, 2004, at 12:26 PM, William Dode wrote:
James Y Knight <foom@fuhm.net> writes:
...
3) For distribution: twisted.web2 depends on Nevow (for static directory listings, and error pages), so will be distributed with it. Nevow recommends Twisted but does not require it for simple cases (e.g. flattening to a string from a CGI script), so there can also be a limited-functionality distribution of standalone Nevow.
What about the wsgi spec ? http://www.python.org/peps/pep-0333.html
It could be great if the server could not depend on any framework (i mean nevow).
The server would depend on Nevow in order to generate the HTML for various automatically generated pages, such as directory listings. I suggested to James that having a standalone twisted.web server that didn't depend on Nevow and wasn't capable of generating any pages itself would be possible, but James didn't think it was worth the effort.
I thought that the current spliting plan was interesting to can release the differents parts independantly. I imagine that twisted.web2 will not move so much than nevow and it's important to don't update a server when we don't need. I hope we'll also find a solution for the wsgi spec, goto the web-sig list... -- William - http://flibuste.net
participants (3)
-
Donovan Preston -
James Y Knight -
William Dode