Mailman 3 October 2004 - Twisted-web

[PATCH] twisted.web2 support for gzip stream in/out, this time for real
by Valentino Volonghi 05 Oct '04

05 Oct '04

The patch is attached and should work. At least this is what I can see from the tests I wrote. Attached there are also the 2 files needed for this patch to work. Both files should go in twsited/web2, but since they can be useful to other protocols too maybe there is a better location. -- Valentino Volonghi aka Dialtone Now running FreeBSD 5.3-beta6 Blog: http://vvolonghi.blogspot.com Home Page: http://xoomer.virgilio.it/dialtone/

1 0

nevow/stackless bugs -- patch
by Peter Hunt 03 Oct '04

03 Oct '04

I've been trying to run stan on stackless when I ran into a problem with render methods. Stackless uses "stackless.instancemethod" instead of "__builtin__.instancemethod", and "stackless.function" instead of "__builtin__.function". Attached is a svn diff patch of the changes I made to __init__.py in order for stan to support stackless properly. HTH, Peter Hunt

3 3

[PATCH] twisted.web2 support for gzip stream in/out
by Valentino Volonghi 02 Oct '04

02 Oct '04

Following there's a patch for twisted.web2 to support gzip Content-Encoding of chunks. Thanks to Bob Ippolito for answering immediately to my mail about the permission to include his gzstream work in twisted.web2. Here is his mail: === From: Bob Ippolito <bob(a)redivi.com> To: Valentino Volonghi <dialtone(a)gmail.com> Date: Fri, 1 Oct 2004 20:50:35 -0400 Subject: Re: gzstream integration with twisted Twisted.web2 may use my gzstream. On Oct 1, 2004, at 8:33 PM, Valentino Volonghi wrote: > Hi, > I'm helping out Twisted developers to get twisted.web2 out before > October 31. > > We are going to have a very complete implementation of HTTP 1.1 RFC. > Doing so requires also having a gzip/gunzip stream to handle the > Transfer-Coding. > > I've already talked to foom (the main developer of t.web2) and he > asked for your permission to include gzstream in twisted.web2. > > That would be very helpful and your copyright will be kept entirely. > > Let me now your decision please. Thanks for attention. > > -- > Valentino Volonghi aka Dialtone > Now running FreeBSD 5.3-beta6 > Blog: http://vvolonghi.blogspot.com > Home Page: http://xoomer.virgilio.it/dialtone/ === files needed are attached. and patch following. I'll write also a bug for web2 and attach everything, but it's 3.30 AM now, and I'll do that tomorrow. Comments are welcome. Index: twisted/web2/http.py =================================================================== --- twisted/web2/http.py (revision 11886) +++ twisted/web2/http.py (working copy) @@ -33,6 +33,7 @@ from twisted.web2 import http_headers from twisted.web2 import iweb from twisted.web2 import error +from twisted.web2 import gzstream PERSIST_NO_PIPELINE = 2 @@ -301,6 +302,12 @@ else: return ifrange == self.out_headers.getHeader("last-modified") + def requestContentEncoding(self, name): + if name.lower() != 'gzip': + warnings.warn("WARNING! Currently only gzip content is supported") + return + self.out_headers.setHeader('Content-Encoding', name) + # FIXME: these last 3 methods don't belong here. @@ -347,6 +354,8 @@ partialHeader = '' queued = 0 finishedReading = False + inChunkHandler = False + outChunkHandler = False channel = None request = None @@ -449,13 +458,14 @@ def rawDataReceived(self, data): datalen = len(data) + if datalen < self.length: if not self.finished: - self.request.handleContentChunk(data) + self.inChunkHandler(data) self.length = self.length - datalen else: if not self.finished: - self.request.handleContentChunk(data[:self.length]) + self.inChunkHandler(data[:self.length]) extraneous = data[self.length:] channel = self.channel # could go away from allContentReceived. if not self.chunkedIn: @@ -493,6 +503,18 @@ self.channel.queueRequest(self) request = self.channel.requestFactory(self, self.command, self.path, self.version, self.reqHeaders) + + def decode_gzip(self, chunkHandler): + gunzipper = gzstream.GunzipStream() + gunzipper.write = chunkHandler + return gunzipper + + # Set the client chunkhandler if the content is gzipped. + for encoding in self.reqHeaders.get('Content-Encoding', ()): + if encoding == 'gzip': + self.inChunkHandler = decode_gzip(request.handleContentChunk) + else: + self.inChunkHandler = request.handleContentChunk # Reset header state variables del self.reqHeaders @@ -633,11 +655,22 @@ l.append("\r\n") self.transport.writeSequence(l) + + def encode_gzip(chunkHandler): + gzipper = gzstream.GzipStream() + gzipper.write = chunkHandler + return gzipper + + for encoding in headers.getHeader('Content-Encoding', ()): + if encoding == 'gzip': + self.outChunkHandler = encode_gzip(toChunk) + else: + self.outChunkHandler = toChunk def writeData(self, data): if self.chunkedOut: - self.transport.writeSequence(toChunk(data)) + self.transport.writeSequence(self.outChunkHandler(data)) else: self.transport.write(data) ======= HTH -- Valentino Volonghi aka Dialtone Now running FreeBSD 5.3-beta6 Blog: http://vvolonghi.blogspot.com Home Page: http://xoomer.virgilio.it/dialtone/

1 1