Mailman 3 May 2003 - Twisted

[Twisted-Python] protocol flows
by Clark C. Evans May 24, 2003

May 24, 2003

Ok. Flow has gone through a rather largish growth period in the last week. Improvments include: 1. It now supports protocols, ie, you can use generators to build protocols. This is not complete, nor is it documented well, but that will come in time. Currently it only supports protocols with one callback, but the flow.py module is quite close to supporting N callbacks via flow.Concurrent (which is still being tested and refined). The flow … [View More]support for protocols still has some issues hiding errors. I've done quite a bit to make exceptions visible... but I'm still a twisted newbie, and a few are slipping through the cracks (or rather hiding between the cracks). The process of making user-level errors clearly reported is on-going. 2. It has quite a few speed improvments, overall flow isn't fast beacuse it basically implements cooperative multitasking. Generally, given noop functions, flow is about 5-7x slower than direct generators. However, most items are not noop, so the actual factor in practice is more like 20% to 2x slower. Not a bad price for relatively easy to read code. I'm not sure how many optimizations are possible; one recent optimization speeded up cases whith lots of little results (but slowed larger more infrequent results). This optimization was due to exarkun's suggestion. Anyway, this change I'm certain has introduced bugs, but they will be squashed in time. Of course, writing the whole module in "C" would give a big bonus. However, flow must reach serious stability and a large user base before that is taken on. I think that the speedup from C coding would be quite significant, but not earth-shattering. 3. A new Callback stage was added. This was needed for protocol support, but can be useful standalone. Other stages include a List and String stage. Not that those are horribly useful, but they do help to illustrate what is going on, so I figured they'd be nice to have. 4. Some serious refactorings. Most importantly in this one is a CallLater instruction. flow.Threaded is much faster with this change. If you were following the source, I'd start from the top down again. A few things to note: A. Some of the interface has changed for those not using for-each. In particular, the state functions (isFailure, etc.) are gone as they were horribly slow. Furthermore, stage.result is now stage.results. If you were using next() explicitly or implicitly via the for-loop, then you will have no problems; as this aspect of the interface hasn't changed. B. This module is approaching stability. I don't expect many more changes... so, if you have comments. Please... let's hear them. C. I don't know if/when flow will be added to twisted, so it's still a use-at-yer-risk Short-term plans: I. Get the protocol to support multiple callbacks II. Add more regression tests, especially ones with icky user-level failures to make sure that errors are being propigated. III. Start writing a few 'flow-filters' and such that work with flow protocols. For instance, one filter could be the equivalent of LineReceiver. Perhaps refactor common code into a separate module... IV. Update/improve the documentation, flow.html Here is an echo "flowprotocol": from __future__ import generators import flow from twisted.internet import protocol, reactor PORT = 8392 print "client server" print "------------------- ---------------------" def echoServer(conn): print " connected, now wait for client" yield conn print " client responded" for data in conn: print " received '%s', sending back" % data yield data print " waiting for client again" yield conn print " disconnected" server = protocol.ServerFactory() server.protocol = flow.makeProtocol(echoServer) reactor.listenTCP(PORT,server) def echoClient(conn): print "connect and send" yield "Hello World" # sending data print "waiting for server" yield conn print "received '%s'" % conn.next() print "sending more" yield "Another Line" print "waiting for server" yield conn print "received '%s'" % conn.next() reactor.callLater(0,reactor.stop) print "disconnecting" client = protocol.ClientFactory() client.protocol = flow.makeProtocol(echoClient) reactor.connectTCP("localhost", PORT, client) reactor.run() Which produces... client server ------------------- --------------------- connect and send waiting for server connected, now wait for client client responded received 'Hello World', sending back waiting for client again received 'Hello World' sending more waiting for server received 'Another Line', sending back waiting for client again received 'Another Line' disconnecting disconnected That's it. Clark [View Less]

1 0

[Twisted-Python] mktap web --resource-script
by Tim Allen May 24, 2003

May 24, 2003

I've started playing with Twisted Web as a way of trying to purge the negative emotions I've associated with server-side scripting due to working with ASP every day, but I've hit a bit of a snag. I could just set up a web.tap file to server ~/public_html, but I thought it'd be cool to write the structure of my site in python with .putChild() (so I can put all the cool generation code in Python modules, rather than parse-every-time .rpy files). The basic script I started with looks like … [View More]

1 0

[Twisted-Python] update for MANIFEST.in
by Daniel Waites May 24, 2003

May 24, 2003

apply the following patch to MANIFEST.in in CVS to get "setup.py bdist_rpm" working again. 33,35c33 < recursive-include twisted *.c < recursive-include twisted *.mgp *.html *.fla *.js *.swf < recursive-include twisted/test *.pem *.message --- > Note that RPM 4 (included in recent RedHat distros) will cause an assertion error at the end, because it builds a debuginfo rpm. It still builds the rpms, though, under build/buildblahblah/rpm/RPMS . This is a bug in Python 2.2 distutils.

1 0

[Twisted-Python] SMTP patch
by Anders Hammarquist May 24, 2003

May 24, 2003

Make sure to drop any references to the file object containing the message data.

6 13

Re: [Twisted-Python] SMTP patch
by Magnus Lyckå May 24, 2003

May 24, 2003

Peter Hansen wrote: >Andrew Bennetts wrote: > > > > Oops! As you imply, that should've been "200 <= code < 300". I still > don't > > like the idea of constructing 100 integers, even lazily, just to test > > bounds, though. > > > > Of course, if the status code was left as a string, then you could > simply do > > "if code.startswith('2'):" :) > >if str(code).startswith('2'): ?? It's much faster to do: if code // 100 == 2: … [View More]

1 0

[Twisted-Python] Lore bug with ".html"-stripping
by Christopher Armstrong May 24, 2003

May 24, 2003

Ok, check out the source of http://twistedmatrix.com/documents/howto/web-overview.html (doc/howto/web-overview.html in CVS) Notice that there are two links to "woven.html". Now, check out the source to http://twistedmatrix.com/documents/howto/web-overview , which is the result of lore-izing the .html. Notice that the first link to woven.html has been properly munged to be simply "woven". The one at the bottom of the page, however, hasn't. Anyone have any ideas on why this is happening? --… [View More]

1 0

[Twisted-Python] How to abuse generators, now that we're allowed to play with 2.2
by Bob Ippolito May 23, 2003

May 23, 2003

Here's some code I've been playing with the past two days.. part of a DAAP (Digital Audio Access Protocol, aka iTunes 4.0) client/server framework that I'm working on that's nowhere near releasable at the moment.. However, the gzip/gunzip streaming code and the example of generator abuse may be of immediate use to someone in the meantime, so here you go. HTTPStream expects you to pass in a protocol that will receive the page incrementally. Try passing in headers={'Accept-Encoding':'… [View More]

1 0

[Twisted-Python] LivePage Hello World
by Eric Holm May 23, 2003

May 23, 2003

I'm trying to get a simple LivePage script going, and no luck so far. I started with the example on howto/livepage.html, and added enough bits to get it to run (the only surprise was that Foo.setUp() needs a 3rd, unused argument), and I've attached it at the end of this mail. When I view the source in the browser, there's a bunch of Flash stuff at the bottom (so *that's* how "Javascript opens Socket to server" works...), but when fooHappened() runs, I get a CACHING [<twisted.web.… [View More]

5 10

[Twisted-Python] Authenticated SSL
by Wari Wahab May 22, 2003

May 22, 2003

Hi there, following the echoserver-ssl.py example, I was able to get up to speed and basically running my xmlrpc server on SSL. As usual, twisted makes it easy. Now my problem becomes this: I need to verify that the certificate on the other side (The client) is signed by me (or a known CA). How would one go about such task?

3 5

Re: [Twisted-Python] protocols.smtp patch
by Anders Hammarquist May 22, 2003

May 22, 2003

In a message of Fri, 16 May 2003 10:13:52 +1200, "matiu" writes: >protocols.smtp client creates a temp file for every email it sends on my >windows box. Apply this patch to stop that. Hmm, should we really be closing files in the SMTPClient? I think it should be enough that the SMTPClient drops any reference it has to the file when it's done with it (which I see it doesn't do currently, I'll add that to my patch and post it momentarily). The responsibility for closing (or deleting or … [View More]

1 0