Mailman 3 July 2007 - Twisted

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz 05 Aug '21

05 Aug '21

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |_____| |_____ | | | | @ t w i s t e d m a t r i x . c o m http://www.twistedmatrix.com/~glyph/

4 5

[Twisted-Python] Get IP of connecting client in XMLRPC server
by Remy C. Cool 28 Apr '13

28 Apr '13

Hello, On 18 May 2004, the following message was posted to this mailinglist: Jp Calderone exarkun at divmod.com wrote: >Daniel Newton wrote: > > I have a simple XML-PRC server similar to the example below: > > from twisted.web import xmlrpc, server > > class Example(xmlrpc.XMLRPC): > """An example object to be published.""" > > def xmlrpc_add(self, a, b): > """Return sum of arguments.""" > return a + b def xmlrpc_whatIsMyAddress(self): return self.transport.getPeer().host > > if __name__ == '__main__': > from twisted.internet import reactor > r = Example() > reactor.listenTCP(7080, server.Site(r)) > reactor.run() > > I want to be able to get the address of the client that calls the xmlrpc > method can anyone help me with this? This solution didn't work because 'transport' isn't a property of the Example class. I'm currently in the process of changing from a customized SimpleXMLRPCServer to a twisted XMLRPC server solution and I need to insert the client IP into the attributes passed to the called xmlrpc method. Anyone who knows the answer and is willing to share the info? Regards, Remy

3 3

[Twisted-Python] Reconnecting PB Client
by Don Smith 12 Sep '07

12 Sep '07

Does anyone have some example code for building a reconnecting PB client? This is what I have now: from twisted.spread import pb from twisted.internet import reactor from twisted.python import util events = pb.PBClientFactory() reactor.connectTCP("localhost", 8789, events) queue = events.getRootObject()

3 2

[Twisted-Python] Win32 IOCPReactor?
by Trent Nelson 04 Sep '07

04 Sep '07

I'm new to Twisted. Perusing the documentation, I came across IOCPReactor. It's marked as 'extremely experimental', and it appears it only provides support for TCP events. It seems the last check in to anything IOCPReactor-related was back in 2005. Is it dead? Is anyone using it? Is anyone planning to pick development back up? Were there specific issues with IOCP that impeded development, or just a general lack of people with time, inclination, or Win32-fu? Assuming the IOCPReactor's not feasible for high load production deployment, can anyone comment on how the default select()-based reactor for Win32 fares under high load? Trent.

19 31

[Twisted-Python] Media Resource Control Protocol (MRCP)
by Ivan Uemlianin 04 Sep '07

04 Sep '07

Dear All I am interested in writing an MRCP (Media Resource Control Protocol [1]) server. In particular I want to write an MRCP speech recognition resource server as part of my trefnydd [2] project. I am posting here to ask for advice about the network programming part. I shall also be writing an MRCP client (if only to test the server). All the code will be GPL'd. As far as I understand it, an overview of the networking would go like this: An MRCP session between client and server is initiated using the Session Initiation Protocol (SIP). Client and server negotiate using SIP messages containing SDP message bodies to define the participants and the resources required in the session. After session initiation, two channels are launched: a media channel using RTP to carry binary audio (i.e., speech) data, and a control channel using TCP (control channel messages are text-based and in MRCP format). It seems to me that both client and server software would have to cope with SIP, SDP, RTP, MRCP (message format) and plain old TCP (for the SIP/SDP and MRCP control channel messages). Here are my questions: - Does anyone know of a FOSS project to write an MRCP server? preferably in python. I know of mrcp4j [3]. - My first plan would be to make the MRCP client/server a subclass of an SIP client/server, as SIP is used to set up (and tear down) the session. Does that sound sensible? - Can twisted.protocols.sip be used to write a SIP server? Would it give me everything I need, or would I also need something like pjsip [4] or jain-sip [5] for the low-level stuff? - Have I missed anything? Any advice would be gratefully received. I'll report back with progress. Best wishes Ivan [1] MRCP: http://www.llaisdy.com/static/tech/mrcp/ [2] trefnydd: http://www.llaisdy.com/static/trefnydd/ [3] mrcp4j: "MRCP4J provides a Java API that encapsulates the MRCPv2 protocol and can be used to implement MRCPv2 clients and/or servers." http://mrcp4j.sourceforge.net/ [4] pjsip: http://www.pjsip.org/ [5] jain-sip: https://jain-sip.dev.java.net/

6 10

[Twisted-Python] Handling PBConnectionLost errors
by Daniel Miller 15 Aug '07

15 Aug '07

Hello, Twisted PB sometimes loses its connection to the server. In this case, a PBConnectionLost exception is raised on the client. It would be nice to implement a fail-safe(er) way of calling remote methods that would retry when necessary until the remote method has been called successfully and the result has been returned. Note that this is only necessary when the remote method call should be invoked exactly once on the server (i.e. for POST-like calls that change server state). In the case of GET-like requests, a simpler retry mechanism will do. The motivation for this is based on my experience of using Twisted in an application I am developing. The network communications are all happening on a LAN. The good end of the network is connected directly to a 100Mbps switch at the server. Failures occur more frequently at the other end (my end) of the network, which is connected through a 10/100 hub that is connected to the main switch. I rigged up a quick test with a 1000-request sample size; failures ranged from 28/1000 on the good end of the network to 83/1000 on the bad end of the network. One request consists of a single remote method call through PB. A success was when I got the expected result, a failure was when I got a PBConnectionLost error. The following is pseudo code that I came up with to mitigate the problem. Simple request (GET - repeatedly call method until success or RETRY_LIMIT is reached) Client flow: for x in range(RETRY_LIMIT) invoke remote method without unique call identifier if result is not PBConnectionLost break if result is PBConnectionLost raise server not responding error Server flow: (nothing special, just plain PB) Complex request (POST - server-side method is invoked exactly once) Client flow: use simple retry method to get a unique call identifier from server a timeout value is also sent along to tell the server how long to hold the results of this request for x in range(RETRY_LIMIT) invoke remote method with identifier if return value is not PBConnectionLost break if result is PBConnectionLost raise server not responding error using simple retry method tell server to discard unique call identifier Server flow: receive request for unique call identifier create and store identifier with UNCALLED token schedule identifier to be discarded with timeout value supplied by client return identifier to client receive remote method invocation with unique call identifier branch on value stored with unique call identifier if UNCALLED update identifier with CALLED token invoke method while result is deferred get defer result store COMPLETED token and unique with unique call identifier if there is another invocation WAITING this means the connection was lost signal the WAITING request with the result else return result to client if CALLED store WAITING token with unique identifier (must not overwrite other call tokens) defer until COMPLETED if COMPLETED return result to client if unique call identifier does not exist raise error receive request to discard unique call identifier if identifier exists discard identifier, tokens, and result return True I realize that implementing this would not eliminate network errors. It would simply reduce the likelyhood of failed method calls due to dropped connections. If I have my math correct (I always struggle a bit with statistics), even a RETRY_LIMIT of 2 would reduce the probability of a lost connection to 0.6% at the worst (<0.1% on the good end of the network). I have two questions: 1. Does something like this already exist? 2. Is this a totally stupid idea? (would it be better to improve our physical network than to try to band-aid the problem with something like this?) ~ Daniel

4 8

[Twisted-Python] "MySQL has gone away"
by Pavel Bastov 29 Jul '07

29 Jul '07

Hi all, I'm using MySQL database in my server app. Pretty often after some inactivity I get this message in my logs "MySQL server has gone away". How can I prevent this? TIA -- Good luck, Pavel Bastov xooChat Evangelist and Team Leader http://www.xoochat.com/

6 7

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 28 Jul '07

28 Jul '07

Bug summary ______________________ Summary for 2007-07-22 through 2007-07-29 Bugs opened: 11 Bugs closed: 9 Total open bugs: 997 (+2) |== Type Changes |== Priority Changes |== Component Changes |Defect: +3 |Highest: -4 |Conch: +1 |Enhancement: +4 |Normal: +5 |Core: -4 |Task: -5 |Low: +1 |Lore: +1 |Mail: -1 |Names: +0 |Trial: +2 |Web: +2 |Words: +1 New / Reopened Bugs ______________________ ===== Highest ===== [#2749] Add a utility method to trial to check the type of an object (opened by therve) enhancement trial http://twistedmatrix.com/trac/ticket/2749 ===== Normal ===== [#2747] Make Lore use filepath (opened by gpaci) enhancement lore http://twistedmatrix.com/trac/ticket/2747 [#2748] Update development/policy test-standard docs (opened by gpaci) defect trial http://twistedmatrix.com/trac/ticket/2748 [#2750] HEAD requests support for twisted.web.client (opened by synapsis) enhancement web http://twistedmatrix.com/trac/ticket/2750 [#2751] "gethostbyname.py www.icq.com" returns "DNS lookup failed" (opened by mc) (CLOSED, invalid) defect names http://twistedmatrix.com/trac/ticket/2751 [#2752] t.w.http.Request docstrings (opened by rikyu) (CLOSED, fixed) enhancement web http://twistedmatrix.com/trac/ticket/2752 [#2753] twisted.web WSGI support (opened by rikyu) enhancement web http://twistedmatrix.com/trac/ticket/2753 [#2754] ssh execCommand: closing the connection throws an error (opened by smurfixx) defect conch http://twistedmatrix.com/trac/ticket/2754 [#2755] Allow a Deferred to be passed to callback() (opened by smurfixx) (CLOSED, wontfix) enhancement core http://twistedmatrix.com/trac/ticket/2755 [#2756] digest_uri in sasl_mechanism.py should be encoded (opened by sciasbat) defect words http://twistedmatrix.com/trac/ticket/2756 ===== Low ===== [#2757] give TrafficLoggingFactory a public API to reset its counter (opened by exarkun) enhancement core http://twistedmatrix.com/trac/ticket/2757 Closed Bugs ______________________ ===== Highest ===== [#2743] twisted/test/test_ssl.py shouldn't use setUpClass and tearDownClass (opened by exarkun, closed by exarkun, fixed) task core http://twistedmatrix.com/trac/ticket/2743 [#2744] twisted/test/test_pb.py should use setUp instead of setUpClass and not have any tear down (opened by exarkun, closed by exarkun, fixed) task core http://twistedmatrix.com/trac/ticket/2744 [#2745] twisted/test/test_newcred.py should use setUp and tearDown instead of setUpClass and tearDownClass (opened by exarkun, closed by exarkun, fixed) task core http://twistedmatrix.com/trac/ticket/2745 [#2746] twisted/mail/test/test_mail.py should use setUp and tearDown instead of setUpClass and tearDownClass (opened by exarkun, closed by exarkun, fixed) task mail http://twistedmatrix.com/trac/ticket/2746 [#2742] twisted/test/test_policies.py should use setUp and tearDown instead of setUpClass and tearDownClass (opened by exarkun, closed by exarkun, fixed) task core http://twistedmatrix.com/trac/ticket/2742 ===== Normal ===== [#2751] "gethostbyname.py www.icq.com" returns "DNS lookup failed" (opened by mc, closed by mc, invalid) defect names http://twistedmatrix.com/trac/ticket/2751 [#1171] twistd has no way to gracefully stop the initialization of an application instance if some conditions are not met (opened by Biappi, closed by glyph, invalid) enhancement core http://twistedmatrix.com/trac/ticket/1171 [#2752] t.w.http.Request docstrings (opened by rikyu, closed by therve, fixed) enhancement web http://twistedmatrix.com/trac/ticket/2752 [#2755] Allow a Deferred to be passed to callback() (opened by smurfixx, closed by therve, wontfix) enhancement core http://twistedmatrix.com/trac/ticket/2755 Ticket Lifetime Stats ______________________ Oldest open ticket - [#38] Should be a way to create documentation aliases (since 2003-07-12 16:08:03). Newest open ticket - [#2757] give TrafficLoggingFactory a public API to reset its counter (since 2007-07-28 08:35:32). Mean open ticket age: 494 days, 19:44:22.394626. Median: 382 days, 10:25:27.387605. Standard deviation: 392 days, 21:05:11.606420. Interquartile range: 473 days, 6:25:54. Mean time between ticket creation and ticket resolution: 119 days, 15:18:10.661056. Median: 14 days, 3:38:33. Standard deviation is 223 days, 7:36:11.456637. The interquartile range is 123 days, 11:30:35. Mean time spent in review: 59 days, 23:00:07.364329. Median: 4 days, 8:59:04. Standard deviation: 129 days, 11:41:34.595642. Interquartile range: 27 days, 10:51:04. Mean number of times a ticket is reviewed: 1.81313131313. Median: 1 Standard deviation: 1.38914598171. Interquartile range: 1. Contributor Stats ______________________ In the last 4 weeks, 16 unique ticket reporters 7 unique ticket reviewers 6 unique ticket resolvers In the last 24 weeks, 94 unique ticket reporters 14 unique ticket reviewers 18 unique ticket resolvers In the last 48 weeks, 162 unique ticket reporters 20 unique ticket reviewers 28 unique ticket resolvers

1 0

[Twisted-Python] potential connectSSH workflow
by Paul Swartz 27 Jul '07

27 Jul '07

I've written up a potential workflow for connectSSH, a simplified way of using Twisted Conch. It's available at http://z3p.jot.com/WikiHome/SummerOfCode2007/connectSSH. If those who are interested could take a look, I'd love some feedback. thanks, -p -- Paul Swartz paulswartz at gmail dot com http://z3p.livejournal.com/ AIM: z3penguin

2 7

[Twisted-Python] wsgi thread synchronization
by Phil Christensen 27 Jul '07

27 Jul '07

I'm crossposting because although my use case is in web, this is more of a question about dealing with threads in twisted. I have a decent academic understanding of threads, but I've never really used them in Python or Twisted. I've just moved from a mod_python WSGI container running on prefork apache2 to a container running on twisted.web, and I realized that I now have threading considerations I didn't before. Since my potential synchronization problems are actually quite few, I've been getting by just by using threading.BoundedSemaphore to manage a couple of thread-sensitive areas of the web app. However, I already have a dependency on Twisted core, so it occurs to me that there are probably nicer, twistier ways of doing this. For example, I have a section of code like: host_tree = {} host_tree_lock = threading.BoundedSemaphore def get_application(env): global host_tree, host_tree_lock host = env.get('HTTP_HOST', env['SERVER_NAME']) host_tree_lock.acquire() if not(host in host_tree): _load_plugins(env) host_node = host_tree[host] if not(host_node.has_path(env['REQUEST_URI'])): _load_plugins(env) app = host_node.get_data_at(env['REQUEST_URI']) host_tree_lock.release() return app The only variable shared between threads is the `host_tree` global (the `env` variable is the WSGI environment, which is unique per thread, and the resulting `app` object is immutable). This works for me and is sufficient for my needs, but I'd appreciate a sanity check from anyone with some thread karma to spare. Thanks again, -phil

2 1