On Tue, 03 Jul 2007 19:52:06 -0000, glyph@divmod.com wrote:
If anyone out there shares his frustration and would like to help us along towards a supported web2 release, or any other part of Twisted for that matter, just have a look at the Twisted tracker and steal a ticket from some of the obviously overcommitted and overworked maintainers. The milestone link I posted is the best start for web2.
I'm already fixing some bugs with the current code as part of the SoC, and I've currently fixed 4 or 5 of them (the last one waiting for review in branch 1895). I'm currently starting, or I'm hoping to start, work on no-stream-1937 branch...
Many of those tickets are underspecified, but don't let that stop you. If you find a ticket that you think sounds interesting but you don't understand how to proceed on it, feel free to comment and ask for clarification - it is often far easier for a maintainer to respond to a prompt for some answers to questions than to drive tests, documentation, and code all the way through the review process.
Since you kindly offered for clarifications: I see one problem with the current IConsumer/IProducer API and it's returning deferreds when a producer finished its job. The current IConsumer API uses registerProducer() but doesn't allow to return anything from the method, and in fact this is what the transport does. Unfortunately it would be useful to deal with the finish event of producing to a consumer for many reasons, like you don't know in advance when the produced content will finish. In Twisted this problem was solved, partially, by FileSender which uses a beginFileTransfer(fobj, consumer) method to return a deferred that fires at the end of the file object. Unfortunately this is not really a reliable method since it's not part of the IProducer interface and any producer might use whatever method they like and, worst of all, I'd anyway need to rewrite or add a method to FileSender because I can't really use beginFileTransfer() for a MemoryBufferProducer() object. One of the needs is the chance to chain Producers/Consumers to do input filtering and output filtering. While the first could also be avoided (meaning that the current implementation is good enough IMHO), output filtering is actually pretty cool to have since it allows the user to dynamically add behavior to a request's response like it's currently done. Taking from JP's consumer-sketch.py code: def _chain(hook, chainedElements): assert len(chainedElements) >= 2 elements = iter(chainedElements) producer = elements.next() if isinstance(producer, Deferred): producer = yield producer for ele in elements: if isinstance(ele, Deferred): ele = yield ele finishedProducing = hook(producer, ele) producer = ele yield finishedProducing _chain = defgen(_chain) def forwardHook(producer, consumer): return producer.produceTo(consumer) def chainOneWay(*x): """ chainOneWay(producer,[chainedElement1,...,chainedElementN],consumer)->Deferred Connect a producer to a consumer/producer to a consumer/producer to a ... ETC For example:: a = FileSender('filename') b = GzipCompressor() c = ChunkEncoder() d = TCPTransport() finished = chainOneWay(a, b, c, d) def cbSentChunkedGzippedFile(ign): print 'Yaaaaaaaaaaaaay' d.loseConnection() finished.addCallback(cbSentChunkedGzippedFile) Will do the obvious thing. """ return _chain(forwardHook, x) This would be a really cool way to solve the problem and would also present a pretty clean and simple API. My question then is: how should I proceed? Clearly using this new API is hard because everything else uses the current one, but the current one lacks a fairly important functionality. Providing adapter in web2 for current producer/consumer stuff might seem overkill and not very clean but might be a solution. Introducing this new API in Twisted would be a nightmare. Is there any way to implement this using the current IProducer/IConsumer API? I seek advice and some code examples if anyone is willing to spend his mind on the problem a bit.