Following there's a patch for twisted.web2 to support gzip Content-Encoding of chunks. Thanks to Bob Ippolito for answering immediately to my mail about the permission to include his gzstream work in twisted.web2. Here is his mail: === From: Bob Ippolito <bob@redivi.com> To: Valentino Volonghi <dialtone@gmail.com> Date: Fri, 1 Oct 2004 20:50:35 -0400 Subject: Re: gzstream integration with twisted Twisted.web2 may use my gzstream. On Oct 1, 2004, at 8:33 PM, Valentino Volonghi wrote:

Hi, I'm helping out Twisted developers to get twisted.web2 out before October 31.

We are going to have a very complete implementation of HTTP 1.1 RFC. Doing so requires also having a gzip/gunzip stream to handle the Transfer-Coding.

I've already talked to foom (the main developer of t.web2) and he asked for your permission to include gzstream in twisted.web2.

That would be very helpful and your copyright will be kept entirely.

Let me now your decision please. Thanks for attention.

-- Valentino Volonghi aka Dialtone Now running FreeBSD 5.3-beta6 Blog: http://vvolonghi.blogspot.com Home Page: http://xoomer.virgilio.it/dialtone/ ===

files needed are attached. and patch following. I'll write also a bug for web2 and attach everything, but it's 3.30 AM now, and I'll do that tomorrow. Comments are welcome. Index: twisted/web2/http.py =================================================================== --- twisted/web2/http.py (revision 11886) +++ twisted/web2/http.py (working copy) @@ -33,6 +33,7 @@ from twisted.web2 import http_headers from twisted.web2 import iweb from twisted.web2 import error +from twisted.web2 import gzstream PERSIST_NO_PIPELINE = 2 @@ -301,6 +302,12 @@ else: return ifrange == self.out_headers.getHeader("last-modified") + def requestContentEncoding(self, name): + if name.lower() != 'gzip': + warnings.warn("WARNING! Currently only gzip content is supported") + return + self.out_headers.setHeader('Content-Encoding', name) + # FIXME: these last 3 methods don't belong here. @@ -347,6 +354,8 @@ partialHeader = '' queued = 0 finishedReading = False + inChunkHandler = False + outChunkHandler = False channel = None request = None @@ -449,13 +458,14 @@ def rawDataReceived(self, data): datalen = len(data) + if datalen < self.length: if not self.finished: - self.request.handleContentChunk(data) + self.inChunkHandler(data) self.length = self.length - datalen else: if not self.finished: - self.request.handleContentChunk(data[:self.length]) + self.inChunkHandler(data[:self.length]) extraneous = data[self.length:] channel = self.channel # could go away from allContentReceived. if not self.chunkedIn: @@ -493,6 +503,18 @@ self.channel.queueRequest(self) request = self.channel.requestFactory(self, self.command, self.path, self.version, self.reqHeaders) + + def decode_gzip(self, chunkHandler): + gunzipper = gzstream.GunzipStream() + gunzipper.write = chunkHandler + return gunzipper + + # Set the client chunkhandler if the content is gzipped. + for encoding in self.reqHeaders.get('Content-Encoding', ()): + if encoding == 'gzip': + self.inChunkHandler = decode_gzip(request.handleContentChunk) + else: + self.inChunkHandler = request.handleContentChunk # Reset header state variables del self.reqHeaders @@ -633,11 +655,22 @@ l.append("\r\n") self.transport.writeSequence(l) + + def encode_gzip(chunkHandler): + gzipper = gzstream.GzipStream() + gzipper.write = chunkHandler + return gzipper + + for encoding in headers.getHeader('Content-Encoding', ()): + if encoding == 'gzip': + self.outChunkHandler = encode_gzip(toChunk) + else: + self.outChunkHandler = toChunk def writeData(self, data): if self.chunkedOut: - self.transport.writeSequence(toChunk(data)) + self.transport.writeSequence(self.outChunkHandler(data)) else: self.transport.write(data) ======= HTH -- Valentino Volonghi aka Dialtone Now running FreeBSD 5.3-beta6 Blog: http://vvolonghi.blogspot.com Home Page: http://xoomer.virgilio.it/dialtone/