Mailman 3 April 2008 - Twisted

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 27 Apr '08

27 Apr '08

1 0

[Twisted-Python] Moving from PB to other RPC protocols
by Justin Mazzola Paluska 25 Apr '08

25 Apr '08

Hi, I’m using PB to connect together various servers in my application. In the application clients connect to a “master” server, which then farms out work to “worker” servers. The master server exports PB objects with two types of functions: (1) accessors that return PB Cacheables and (2) mutators that change the state of the server (and consquently alter any outstanding Cacheables). In the past all of our client programs (some wx, a GTK client, and some command line tools) were all written in Python. Now, however, we want to integrate two new applications with our system. One is a web application with a developer who’d rather write a WSGI application and use some blocking RPC method and the other is a large C++ application that can’t be re-written in Python. How do we get everyone to communicate? In order to support these new applications, it looks like our master server will need to offer two interfaces: one PB and one something else. - I imagine that we can choose any old RPC method (XML-RPC, json-rpc, random protocol over HTTP) to implement our mutator functions. In the Twisted mailing list archives, most people recommend XML-RPC, though json-rpc looks intruiging (especially for the web stuff). - Is there any common RPC mechanism with anything like Cacheables? I loathe to have these new applications polling the server all the time when they could just be waiting for something. Thanks, —Justin

1 0

Re: [Twisted-Python] Problem exiting script that calls a func returning a called deferred
by Terry Jones 25 Apr '08

25 Apr '08

Hi Amaury > did you try something like: > > def stop(x): > reactor.callLater(0, reactor.stop) > > This ensures that the reactor is started before you tell it to stop. That's perfect, thanks. Terry

1 0

[Twisted-Python] Problem exiting script that calls a func returning a called deferred
by Terry Jones 25 Apr '08

25 Apr '08

>From searching and reading the list archives, I know the following comes up under various guises, and although I think I understand what's going on, I don't see a nice solution. I'm trying to write a standalone script. It calls externalFunc, a function someone else wrote that returns a deferred. My script could look something like this: from twisted.internet import reactor from twisted.python import log if __name__ == '__main__': def ok(result): print "ok called with", result def nok(failure): print "nok called with", failure return failure def stop(x): reactor.stop() d = externalFunc() d.addCallback(ok) d.addErrback(nok) d.addBoth(stop) d.addErrback(log.err) reactor.run() And that works fine. Now suppose I want to do some testing in which I swap out the external function for a library I wrote myself (or for another implementation). In the testing library, externalFunc is simplified and uses defer.success() to return its result. In this case the script raises RuntimeError, "can't stop reactor that isn't running" because the deferred that comes back from externalFunc has already been called. So when I add the call/errbacks, they are fired right away and the reactor isn't running when the stop function tries to stop it. And script never exits because it then starts the reactor. At this point there are various contortions I can try. Catching the RuntimeError works, but the script never exits. Catching the RuntimeError and then calling sys.exit doesn't work because sys.exit raises SystemExit and that is caught in twisted/internet/defer.py _runCallbacks. Supposing I had more control, the calling function could create the deferred, pause it, pass it to externalFunc, add callbacks, then unpause. But that doesn't work either as the callbacks fire as soon as unpause is called, and you get the reactor not started error, followed by no exit as you then start the reactor. Testing reactor.running before trying to stop it in stop gets rid of the error, but the reactor is then started and not stopped. The only bulletproof way to deal with this that I've come up with is to call self.fireSystemEvent("shutdown") but that feels wrong. Several solutions seem possible with changes to Twisted: don't catch SystemExit; give reactor.stop a (default False) keyword arg to tell it to ignore a stopped reactor; stop deferreds from firing call/err backs if they are paused (IMO it's a bit non-paradigmatic to have call/errback events firing when the reactor isn't even running), etc. I suppose I'm just not doing things right. Can someone tell me how I should be doing this? Terry

2 1

[Twisted-Python] Forcing telnet connections to character mode using twisted.conch.telnet.Telnet
by Bryan Schmersal 24 Apr '08

24 Apr '08

I've been using twisted for about 36 hours now and have an application put together, but in order to get it working just the way I want it to, I want to force the other end of the telnet connection to go into character-at-a-time mode. I haven't seen any examples of setting IAC options, but perhaps I haven't been looking in the proper places. Does anyone have any hints? Bryan

2 2

[Twisted-Python] twisted.web2's equivalent to setHeader?
by inhahe 24 Apr '08

24 Apr '08

It took some searching, but I found something that states directly that Twisted doesn't support pipelining ( http://www.dalkescientific.com/writings/diary/archive/2006/08/28/levels_of_… ". Twisted doesn't support HTTP/1.1 pipelining and neither does urllib.") I think I read elsewhere that web2 does support pipelining, but I'm not sure if FireFox is using pipelining with twisted.web2; it uses heuristics to determine whether to use pipelining or not (provided you enabled pipelining) based on the server type. Wikipedia says most servers support pipelining so I assume Apache does, so I was thinking I could make web2 pose as an Apache server by doing setHeader("Server", "Apache/2.2.8(Unix)"), but I get this error.. ctx.setHeader("Server", "Apache/2.2.8(Unix)") exceptions.AttributeError: 'Request' object has no attribute 'setHeader' so how do I set a header using web2?

3 2

[Twisted-Python] Strange behavior when transferring large strings
by Gabriel Rossetti 24 Apr '08

24 Apr '08

Hello, I have been trying to use the producer/consumer paradigm to transfer large strings. I've been having trouble because I have strange things going on, but I now suspect it isn't the P/C that's not working, because it works fine with smaller strings than the ones I'm having trouble with, so I suspect the problem comes from elsewhere. What happens is it starts off fine, but then the destination at some point only gets part of the string but the source has already sent everything, closed it's transport and called stopProducing(). Could there be some internal buffering problem, like I'm sending too much data at too high of a throughput, or maybe something like some sort of max time a socket can stay open or something? If I tell it to split the string up into smaller parts, it doesn't change anything, if I make the string smaller, whatever the size of each chunk, it transfers fine. Thanks, Gabriel

3 6

[Twisted-Python] pipelining with FireFox and twisted.web2
by inhahe 24 Apr '08

24 Apr '08

it wasn't easy to find, but i finally found something that states directly that Twisted http://www.dalkescientific.com/writings/diary/archive/2006/08/28/levels_of_…

1 1

[Twisted-Python] Could this be an error in pcp.py?
by Gabriel Rossetti 24 Apr '08

24 Apr '08

Hello, I was looking at pcp.py and I noticed this : def resumeProducing(self): self.paused = False if self._buffer: data = ''.join(self._buffer) bytesSent = self._writeSomeData(data) if bytesSent < len(data): unsent = data[bytesSent:] assert not self.iAmStreaming, ( "Streaming producer did not write all its data.") self._buffer[:] = [unsent] else: self._buffer[:] = [] else: bytesSent = 0 ... in the ProducerConsumerProxy class, self._writeSomeData(data) returns the length of what was written, but in the test right afterwards bytesSent is compared to len(data), but they both have the same value, shouldn't it be "if bytesSent < len(self._buffer):" instead? Gabriel

2 6

[Twisted-Python] Re: therve changed u'Changes to TwistedNames to make it support NAPTR records'
by Roland Hedberg 23 Apr '08

23 Apr '08

trac(a)twistedmatrix.com wrote: > Ticket u'Changes to TwistedNames to make it support NAPTR records' > changed by therve <therve(a)free.fr>: > > First, thanks a lot for your contribution. I'm sorry we lost some > much time handling it. I think it's already in pretty good shape, but > it would need more documentation and tests to land in trunk. Are you > willing to help on these topics? Could do that! What would the process be ? -- Roland

1 0

Twisted April 2008