Mailman 3 April 2004 - Twisted

Re: [Twisted-Python] why deferred.setTimeout is not my favorite API method
by exarkun＠divmod.com 18 Apr '04

18 Apr '04

On Sun, 18 Apr 2004 10:25:34 +1000, Andrew Bennetts <andrew-twisted(a)puzzling.org> wrote: > > [snip] > > I don't see why allowing users of Deferreds to just say > "d.setTimeout(5)" is a bad thing? > > [snip] I tend to prefer "foo(x, y, timeout=z)" over "foo(x, y).setTimeout(z)". As both require explicit support from the implementor of "foo" (as you correctly note is the only way .setTimeout() can actually work), it seems unnecessary to special-case this form of failure in Deferreds. I cast my vote for the former. Jp

1 0

Re: [Twisted-Python] why deferred.setTimeout is not my favorite API method
by exarkun＠divmod.com 17 Apr '04

17 Apr '04

On Sat, 17 Apr 2004 18:22:50 -0400, Jonathan Simms <slyphon(a)twistedmatrix.com> wrote: > > [snip] > > ------------------------------------------------------ > > d = iReturnADeferred() > > def onTimeout(): > handleTimeoutCondition() > > delayedCall = reactor.callLater(timeoutLen, onTimeout) > > def success(value): > handleSuccess() > > def nonTimeoutFailure(reason): > handleErrorCondition() > > def cancelTimeout(val): > if delayedCall.active(): > delayedCall.cancel() return val > > d.addBoth(cancelTimeout) > d.addCallback(success) > d.addErrback(nonTimeoutFailure) > > ------------------------------------------------------- While I think the above is very close to being correct (it even fooled me into thinking it was correct for several minutes, during which I composed an alternate email expounding on how we might simplify users' lives by extracting the common portions of the above into a helper function), ultimately even it does not solve all the issues relating to timing out Deferreds. The problem is simply that "onTimeout" in the above code cannot be correctly defined by anyone except the person who defined "iReturnADeferred". The above pattern may be useful for framework implementors, but not users. Framework implementors typically (though not always) understand how to handle timeout issues, so this is of little help. I think that users asking for help with setTimeout or with timing out Deferreds need to be handled in one of two ways: 1) Direct them to APIs related to the features they wish to use with timeouts that already support timeouts directly (ie, not through setTimeout calls on the Deferreds they return). Examples of such APIs include twisted.web.client.getPage and twisted.names.client.lookupAddress. 2) If APIs for the feature they wish to use do not support timeouts, explain this. Recommend that the user submit a patch or open an issue on the bug tracker, and/or apologize to them that Twisted is not yet perfect in every respect. > [snip] > > It makes no sense to leave a sharp, pointy thing lying around for users > to hurt themselves with, and to constantly tell them "DON'T USE THAT". > I think that deprecating setTimeout is a good thing, even if several years must pass before we can actually remove it. In addition to the reasons Jonathan gave, I will add this: setTimeout() is the only part of Deferreds that tie them to twisted.internet. If it is removed, defer.py could (other issues, such as backwards compatibility and user consternation aside) be moved to the twisted.python package. It also makes it that much easier to use Deferreds with a non-Twisted reactor. Jp

3 2

[Twisted-Python] On splitting Twisted into subprojects
by Mary Gardiner 17 Apr '04

17 Apr '04

Note: Cross-posted to twisted-web and twisted-python. Decide between them for your follow-ups please :) This is intended to spark a little more discussion on splitting Twisted into subprojects, as discussed on Aussie-time on IRC (Jerub, jml, spiv and myself). It seems like the twisted-web project are planning to split off fairly soon: see http://twistedmatrix.com/pipermail/twisted-web/2004-April/000312.html It's pretty likely that this split will go ahead without any kind of imposed-from-above decisions about where the website will go, what the release cycle will look like, where the docs will move to, what the status of shared docs will be (I tend to focus on docs...) The nightmare scenario for splitting twisted into subprojects is that every subproject develops its own release code, release procedures, testing framework, website, documentation style, community, policies... etc etc. And shortly after that we all spiral into the sun or something. There appeared to us to be three possible ways of avoiding this: 1. Luck 2. Release code, release procedures, testing framework, website, documentation style, community, policies... imposed from above 3. Twisted Web (or whatever other project splits first) acting as a "model split": evolving release mechanisms, policy etc etc that are fundamentally sane and wonderful so that noone would think of doing things any other way. Since 3 looks like being the most achievable, this is basically an appeal to the twisted-web folk to keep in mind that they should, where possible, try and make twisted-web the Model of a Modern Twisted Subproject. And make it as easy as possible for other subprojects to copy whatever it is that they're going to do. I personally would like to offer my help in documenting subproject procedures as they evolve, and storing that documentation in some kind of prominent place. -Mary

3 6

[Twisted-Python] Minor question wrt threading pools
by Mike C. Fletcher 17 Apr '04

17 Apr '04

I'm sitting down to move more of our application into Twisted and have come across a minor question that I'm thinking would be a good addition to the threading howto. In particular, starting a thread-pool seems to start the threads as non-daemon threads, so that on completion of the main reactor loop the threads are still hanging around (AFAICS, anyway, the app sits hung waiting for the background threads to complete). It would be useful to know: * How to start threads in the threading pool as daemon threads (i.e. will exit when all non-daemon threads exit) * How to force an exit of all threads o How to access the currently active thread-pool (I assume the documented ThreadPool.stop method is what needs to be called, but how to get the ThreadPool object isn't documented AFAICS). Anyway, it's not a show-stopper, I can just kill the app manually after each test run, still, would love to have an answer. Have fun, Mike _______________________________________ Mike C. Fletcher Designer, VR Plumber, Coder http://members.rogers.com/mcfletch/

2 2

Re: [Twisted-Python] pb: async call in remote method
by Brian Warner 16 Apr '04

16 Apr '04

> def remote_getValue(self, someParam) > result = processAsyncRequest(someParam) > return notYetReceivedResult > > Is it possible to somehow defer (funny heh) the response that > remote_getValue should send back to the client until the result from > processAsyncRequest arrives? Yes, and in fact a Deferred is precisely what you use to do it. When a remote_* method returns a Deferred, the PB internals know they should add a callback to it, and they will refrain from sending a response to the client until that Deferred fires. def remote_getValue(self, someParam): d = processAsyncRequest(someParam) return d All you have to do is write your processAsyncRequest() function to create and return a Deferred, and then do something which fires it with d.callback(results) when the answer comes back. Maybe something like this: class RequestHandler: def __init__(self): self.requests = {} self.reqnum = 0 def processAsyncRequest(self, param): reqnum = self.reqnum self.reqnum += 1 d = defer.Deferred() self.requests[reqnum] = d sendRequest(reqnum, param) # send query to someone # assume our "answerReturned" method is called when they respond return d def answerReturned(self, reqnum, results): d = self.requests[reqnum] del self.requests[reqnum] d.callback(results) cheers, -Brian

2 1

[Twisted-Python] pb: async call in remote method
by Sorin C. 16 Apr '04

16 Apr '04

Hi all, I have a client and server that talk to each other through pb. The situation is the following: The client calls a remote method: """ d = obj.callRemote('getValue', someParam) d.addCallback(self.gotValue) """ On the server side: """ class PbValue(pb.Referenceable): def remote_getValue(self, someParam) result = processAsyncRequest(someParam) return notYetReceivedResult """ The problem is that processAsyncRequest is asynchronous, sends a request to another server (not Python) and returns without waiting for response, and so there is no result to return from the remote method back to the client at this point. Is it possible to somehow defer (funny heh) the response that remote_getValue should send back to the client until the result from processAsyncRequest arrives? Thanks, Sorin __________________________________ Do you Yahoo!? Yahoo! Tax Center - File online by April 15th http://taxes.yahoo.com/filing.html

1 0

Re: [Twisted-Python] New user, help needed with raw UDP packets
by exarkun＠divmod.com 16 Apr '04

16 Apr '04

On Fri, 16 Apr 2004 09:34:49 -0400, Itamar Shtull-Trauring <itamar(a)itamarst.org> wrote: >On Fri, 2004-04-16 at 03:00, Richard Taylor wrote: > > > I have been looking at how to subclass usp.Port to get what I want but I have > > come up against a problem that looks like a show stopper to me. It does not > > look like Python's socket library supports SOCK_RAW. I may be mistaken, but I > > think that I need to put the socket into RAW mode to get the incoming headers > > and to set my own headers on the outgoing packets. Without SOCK_RAW I am > > stuffed. > > Were you running as root? That would be a requirement. Moreover, "Python's" socket library is really just a thin wrapper around the platform socket library. Restrictions such as "Raw sockets can only be created by root" are imposed by the platform, not by Python. Similarly, if the platform socket library does not support raw sockets, Python will not support raw sockets. Since you seem to have the SOCK_RAW name in the socket module, I'm guessing that Itamar's guess of privileges being the problem is correct, rather than support being completely missing. Jp

1 0

[Twisted-Python] running local process to share serial port to many tcp clients
by Denis Barmenkov 16 Apr '04

16 Apr '04

Hello! Is there some example/tutorial, how top create twisted application for sharing unique resource? I have some hardware module, which connects to Linux computer via serial port. Simple protocol also exists: after writing some data to device, it think ;), then returns response. And my question is: how can I write separate process, which acts as serial_device proxy? And how to implement interaction between twisted.internet.protocol.Protocol's descendants and this device-sharing protocol? Thank you! Regards, Denis Barmenkov

1 0

[Twisted-Python] explanation of the chatserver.py example
by George Patterson 16 Apr '04

16 Apr '04

I have trying to write a chat server that will selectively send out messages to some clients. Anyway, looking around for a framework to start from and learn more about Python and other modules I stumbled upon chatserver.py twisted.sourceforge.net But I can't find what some of the aspects the MyChat class does. From: http://twisted.sourceforge.net/TwistedDocs-1.2.0/examples/chatserver.py class MyChat(basic.LineReceiver): def connectionMade(self): print "Got new client!" self.factory.clients.append(self) def connectionLost(self): print "Lost a client!" self.factory.clients.remove(self) def lineReceived(self, line): print "received", repr(line) for c in self.factory.clients: c.message(line) def message(self, message): self.transport.write(message + '\n') #end of class If Bob, Jane and Joe are chatting to each other, and Bob sends a string of characters to Jane as a private message, how would I loop through the lineReceived definition for Jane's client ID or IP address or something unique to Jane's connection. I want to change def lineReceived to something like def lineReceived(self, line): print "received", repr(line) for c in self.factory.clients: if c.id==required_id: # ^-- I know this is not valid code c.message("Privmsg". line) #end if #end for Sorry if it is a stupid question but I have been staring long enough at the code. George Patterson

2 1

[Twisted-Python] Re: [Twisted-commits] r10475 - Use writeSequence instead of silly for-loop.
by exarkun＠divmod.com 15 Apr '04

15 Apr '04

On Thu, 15 Apr 2004 13:27:56 -0400, James Y Knight <foom(a)fuhm.net> wrote: > > On Apr 5, 2004, at 10:36 AM, Andrew Bennetts wrote: > > > This is of course quite silly, and I suspect a nice performance > enhancement could be achieved by making FileDescriptor.dataBuffer be a > list of strings, rather than a single string. That would both cause > multiple writes in a row to not allocate new strings, and allow > writeSequence to be implemented sanely. Been there :) FileDescriptor.dataBuffer was a list when I first started hacking around in Twisted's internals. The current implementation is the result of an optimization of that code, and is somewhat (not earth-shatteringly) faster. > > Doesn't look very hard really; looks like you'd only need to touch > write/writeSequence/doWrite. This'd also have the advantage of making > it easier to put a Sendfile-struct in there as well. So pahan's > volunteered to fix this! There are a few other places that need to be touched, but it is certainly doable. The main problem is simply that tcp.py is such a mess. Jp

1 0

Twisted April 2004