Mailman 3 May 2006 - Twisted

[Twisted-Python] Re: [Twisted-commits] r16956 - Branch for 2518bis updates.
by Jean-Paul Calderone 29 May '06

29 May '06

On Mon, 29 May 2006 15:58:04 -0600, Wilfredo Sanchez <wsanchez(a)wolfwood.twistedmatrix.com> wrote: >Author: wsanchez >Date: Mon May 29 15:58:03 2006 >New Revision: 16956 > >Added: > branches/wsanchez/bis/ > - copied from r16955, trunk/ >Log: >Branch for 2518bis updates. > We've stopped using per-user directories in the branches hierarchy. Also, all branch names should be of the form <short descriptive name>-<ticket number>. See http://divmod.org/trac/wiki/UltimateQualityDevelopmentSystem for documentation on the procedure that should be used for all Twisted development. Jean-Paul

1 0

[Twisted-Python] Flow, generators, coroutines etc.
by Dominic Fox 29 May '06

29 May '06

I understand that twisted.flow is no longer maintained, and is not widely considered to represent a good way of writing twisted code. However, I haven't been able to find any explanation of why this approach (using generators to simulate co-operative multitasking) seems to have been abandoned. Is it simply the case that most people writing twisted code didn't find it very useful? Or are there more specific arguments against doing things that way? I'm trying to make the case at work for using twisted for networking things (in spite of my preference for lightweight threads plus sensible concurrency primitives, if Python is the target platform then twisted is probably the best way to go). If I can show some full-threaded code next to some co-operative multitasking code that a) has much the same sort of control flow, but b) scales much better, and doesn't have to worry about subtle concurrency issues, then I think it should go fairly well. If I have to explain about how the event-driven programming model works at the same time, it might not go *so* well... Dominic

6 5

[Twisted-Python] pr0blems with meds expencies, you Are 0n right way
by Grant 29 May '06

29 May '06

1 0

[Twisted-Python] prOblems with meds expencies, you Are On right way
by Adan 29 May '06

29 May '06

1 0

[Twisted-Python] reactor.connectSSL() error handling
by Lorenzo Allegrucci 29 May '06

29 May '06

Hi, I'm writing a distributed application using XML-RPC over SSL in Twisted. I need mutual authentication (server and client) but the XML-RPC Proxy in Twisted doesn't take any SSL context so I rewrote a Proxy by myself passing my ClientContextFactory subclass to reactor.connectSSL. Everything works fine but now I need to handle some possible error conditions that could prevent the connection, e.g wrong certificates, incompatible ciphersuite etc. In other words, I would like to catch these "exceptions" and handle them but I don't know how to do it in the Twisted framework (using Deferred I guess..). Any hint? Thank you.

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com 28 May '06

28 May '06

Bug summary ______________________ Summary for 2006-05-21 through 2006-05-28 Bugs opened: 28 Bugs closed: 19 Total open bugs: 614 (+9) |== Type Changes |== Priority Changes |== Component Changes |Defect: +5 |Highest: -1 |Conch: -2 |Enhancement: +2 |High: +2 |Core: +7 |Task: +2 |Normal: +7 |Lore: +0 |Low: +1 |Mail: +0 |Names: +0 |News: +0 |Release Management: +0 |Runner: +0 |Web: +0 |Web2: +2 |Website: +1 |Words: +1 New / Reopened Bugs ______________________ ===== Highest ===== [#1737] Release Twisted conch 0.7.0 (opened by radix) (CLOSED, fixed) task conch http://twistedmatrix.com/trac/ticket/1737 [#1743] Release Twisted Lore 0.2.0 (opened by radix) (CLOSED, fixed) task lore http://twistedmatrix.com/trac/ticket/1743 [#1745] Release Twisted Runner 0.2.0 (opened by radix) (CLOSED, fixed) task runner http://twistedmatrix.com/trac/ticket/1745 [#1747] RELEASE "TWISTED" 2.4.0!!!! (opened by radix) (CLOSED, fixed) enhancement release management http://twistedmatrix.com/trac/ticket/1747 [#1749] update the docs on the web site for 2.4.0 (opened by radix) (CLOSED, fixed) task website http://twistedmatrix.com/trac/ticket/1749 ===== High ===== [#1742] Add stdio halfclose tests (opened by exarkun) task core http://twistedmatrix.com/trac/ticket/1742 [#1752] t.w2's header tests that use runRoundtripTest and requiredGeneratedElements are invalid. (opened by dreid) defect web2 http://twistedmatrix.com/trac/ticket/1752 [#1753] ConnectionPool.good_sql is not good SQL as far as Oracle is concerned (opened by itamarst) enhancement core http://twistedmatrix.com/trac/ticket/1753 [#1754] adbapi has blanket except: with no logging (opened by itamarst) enhancement core http://twistedmatrix.com/trac/ticket/1754 [#1755] t.w2.twscgi.SCGIClientProtocol causes an exception to be raised in t.w2.log._LogByteCounter (opened by dreid) defect web2 http://twistedmatrix.com/trac/ticket/1755 ===== Normal ===== [#1729] release twisted core 2.4.0 (opened by radix) (CLOSED, fixed) task core http://twistedmatrix.com/trac/ticket/1729 [#1730] twisted.test.test_tcp.HalfClose3TestCase.testReadNotificationRaises fails intermittently (opened by radix) defect core http://twistedmatrix.com/trac/ticket/1730 [#1731] Release twisted names 0.3.0 (opened by radix) (CLOSED, fixed) task names http://twistedmatrix.com/trac/ticket/1731 [#1732] Release twisted mail 0.3.0 (opened by radix) (CLOSED, fixed) task mail http://twistedmatrix.com/trac/ticket/1732 [#1733] Release Twisted words 0.4.0 (opened by radix) (CLOSED, fixed) task words http://twistedmatrix.com/trac/ticket/1733 [#1734] twisted.words.service has NameErrors (opened by radix) defect words http://twistedmatrix.com/trac/ticket/1734 [#1735] skip twisted.words' msn tests when twisted.web is not available (opened by radix) task words http://twistedmatrix.com/trac/ticket/1735 [#1736] Release Twisted web 0.6.0 (opened by radix) (CLOSED, fixed) task web http://twistedmatrix.com/trac/ticket/1736 [#1738] Failure.__getstate__() does not clean self.value() (opened by moof) defect core http://twistedmatrix.com/trac/ticket/1738 [#1739] (patch) Removing system event triggers while firing does the wrong thing. (opened by antony) defect core http://twistedmatrix.com/trac/ticket/1739 [#1740] twisted/test/stdio_test_halfclose.py typo (opened by rikyu) (CLOSED, fixed) defect core http://twistedmatrix.com/trac/ticket/1740 [#1741] threadCallQueue loop can grow list forever (opened by ghazel) defect core http://twistedmatrix.com/trac/ticket/1741 [#1744] listen invalid argument (opened by ghazel) defect core http://twistedmatrix.com/trac/ticket/1744 [#1746] Release Twisted News 0.2.0 (opened by radix) (CLOSED, fixed) task news http://twistedmatrix.com/trac/ticket/1746 [#1748] Using twisted.cred.credentials results in deprecation warnings in Twisted 2.4 (opened by szumo) defect core http://twistedmatrix.com/trac/ticket/1748 [#1750] Solaris tar does not work with Twisted tarball (opened by damon_register) (CLOSED, wontfix) defect website http://twistedmatrix.com/trac/ticket/1750 [#1751] Make macro or plugin or something so project pages automatically display latest releases (opened by itamarst) enhancement website http://twistedmatrix.com/trac/ticket/1751 ===== Low ===== [#1756] Add preliminary support for UTF-8 to twisted.conch.insults (opened by radix) enhancement conch http://twistedmatrix.com/trac/ticket/1756 Closed Bugs ______________________ ===== Highest ===== [#1695] raising an exception from datagramReceived should not kill a listening UDP Port (opened by radix, closed by exarkun, fixed) enhancement core http://twistedmatrix.com/trac/ticket/1695 [#1737] Release Twisted conch 0.7.0 (opened by radix, closed by exarkun, fixed) task conch http://twistedmatrix.com/trac/ticket/1737 [#1745] Release Twisted Runner 0.2.0 (opened by radix, closed by exarkun, fixed) task runner http://twistedmatrix.com/trac/ticket/1745 [#1749] update the docs on the web site for 2.4.0 (opened by radix, closed by exarkun, fixed) task website http://twistedmatrix.com/trac/ticket/1749 [#1743] Release Twisted Lore 0.2.0 (opened by radix, closed by radix, fixed) task lore http://twistedmatrix.com/trac/ticket/1743 [#1747] RELEASE "TWISTED" 2.4.0!!!! (opened by radix, closed by radix, fixed) enhancement release management http://twistedmatrix.com/trac/ticket/1747 ===== High ===== [#635 ] Patch for py2exe 0.5 support for tap2ntsvc (opened by jml, closed by moonfallen, invalid) defect conch http://twistedmatrix.com/trac/ticket/635 [#1013] Windows batch file arguments (opened by nicktrout, closed by moonfallen, invalid) defect conch http://twistedmatrix.com/trac/ticket/1013 [#1016] setup.py all --help causes infinite loop (opened by justinj, closed by moonfallen, invalid) defect conch http://twistedmatrix.com/trac/ticket/1016 ===== Normal ===== [#1729] release twisted core 2.4.0 (opened by radix, closed by radix, fixed) task core http://twistedmatrix.com/trac/ticket/1729 [#1731] Release twisted names 0.3.0 (opened by radix, closed by radix, fixed) task names http://twistedmatrix.com/trac/ticket/1731 [#1732] Release twisted mail 0.3.0 (opened by radix, closed by exarkun, fixed) task mail http://twistedmatrix.com/trac/ticket/1732 [#1714] Update Words' NEWS for 0.4 release (opened by ralphm, closed by exarkun, fixed) enhancement words http://twistedmatrix.com/trac/ticket/1714 [#1733] Release Twisted words 0.4.0 (opened by radix, closed by exarkun, fixed) task words http://twistedmatrix.com/trac/ticket/1733 [#1736] Release Twisted web 0.6.0 (opened by radix, closed by radix, fixed) task web http://twistedmatrix.com/trac/ticket/1736 [#1740] twisted/test/stdio_test_halfclose.py typo (opened by rikyu, closed by exarkun, fixed) defect core http://twistedmatrix.com/trac/ticket/1740 [#1746] Release Twisted News 0.2.0 (opened by radix, closed by radix, fixed) task news http://twistedmatrix.com/trac/ticket/1746 [#1750] Solaris tar does not work with Twisted tarball (opened by damon_register, closed by glyph, wontfix) defect website http://twistedmatrix.com/trac/ticket/1750 [#1725] selectreactor 'winsock error' (opened by ghazel, closed by glyph, wontfix) defect core http://twistedmatrix.com/trac/ticket/1725

1 0

[Twisted-Python] Twisted 2.4.0 released
by Christopher Armstrong 27 May '06

27 May '06

Twisted is an event-based framework for internet applications which works on Python 2.3.x and 2.4.x. The 2.4.0 release includes features and fixes for the various parts of Twisted, including Internet, Conch, Web, Mail, Names, and more. Hit http://twistedmatrix.com/trac/wiki/TwistedProject to see what's new and to get the latest downloads, including tarballs and Windows installers. For general information about Twisted, see the web site at http://twistedmatrix.com/. Thanks to Jean-Paul Calderone for helping a lot to get this release out the door. -- Christopher Armstrong International Man of Twistery http://radix.twistedmatrix.com/ http://twistedmatrix.com/ http://canonical.com/

1 0

[Twisted-Python] deferToThread - supported alternative to the (deprecated) setTimeout method
by Paul Moore 26 May '06

26 May '06

I have a program where I think that what I want to use is deferToThread and setTimeout. However, the setTimeout method is marked as deprecated - but I can't find a good supported alternative. Can anyone help? The situation I have is: I need to do a large-ish number (50-100) of blocking calls in parallel, and collect the results or any errors. In itself, this seems like a suitable use for deferToThread (defer each call, and collect the results in the deferred callback/errback). The problem is that in rare cases, the blocking call can block indefinitely. In this case, I need to make the call time out. However, the underlying API offers no way to time out the call, so I have to do this externally. A quick prototype seems to work OK, using setTimeout to force a timeout on the deferred, but it generates deprecation warnings for the setTimeout call. What should I be using to achieve this effect? Thanks, Paul. -- I remember being impressed with Ada because you could write an infinite loop without a faked up condition. The idea being that in Ada the typical infinite loop would normally be terminated by detonation. -- Larry Wall

6 6

[Twisted-Python] XML-RPC server
by Kelly Vincent 26 May '06

26 May '06

I am quite new to Twisted and I'm having some trouble getting my XML-RPC server to do exactly what I want. We have a relatively simple goal: create an XML-RPC server that will accept a request for a score, call a function that computes that score, and then return the score. The two catches are that 1. the client is actually a .NET web server rather than a Twisted client, and 2. we could have thousands of requests come in relatively simultaneously (though more likely not greater than a few hundred), likely with lots of downtime between these periods of high activity. We are planning to create a look-up table to deal with some of the more common calculations, but I still need to be able to run the other calculations non-sequentially. I know that using a Deferred object is the normal solution, but I'm not entirely sure if it could work here without threads. I read somewhere that when the function call is not from another Twisted application, it's not possible to use the Deferred object. Is this actually the case? Or would it work in my situation since all of the "deferring" would be occurring within the server itself? My current solution involves using threads and a Deferred object. Is this the best way to do this? Can it be done without threads? If so, how and is that a better solution than the one with threads? Here is my sample code, which does a simple calculation rather than the real one I will be using. Both my Twisted testing client and the .NET web server are able to talk to this server and get the correct results back. from twisted.web import resource, server, xmlrpc from twisted.internet import defer, threads from twisted.python import threadable threadable.init() class ScoreServer(xmlrpc.XMLRPC): def xmlrpc_calculateScore(self,params): t = params[0] d = threads.deferToThread(self.calc,t) val = d.addCallback(self.handleSuccess) d.addErrback(self.handleFailure) print t, d return val def calc(self,t): import time time.sleep(t) result = [t,t*10] return result def handleSuccess(self,t): print 'succeeded', t return t def handleFailure(self): print 'failed miserably' return ['Error'] if __name__ == "__main__": from twisted.internet import reactor root = resource.Resource() root.putChild('RPC2', ScoreServer()) reactor.listenTCP(8082, server.Site(root)) reactor.run()

3 2

[Twisted-Python] web.client blowing up on non-fully qualified 301s
by Keith Dutton 25 May '06

25 May '06

Hello, I have run into an odd problem. I am not sure if it is my issue or Twisted: any help would be appreciated. Under at least some circumstances, twisted.web.client seems to 1) not be able to follow a 301, and 2) throw an unhandled exception, when trying to follow a 301. A specific example and resulting error is given below: from twisted.internet import defer from twisted.web import client from twisted.internet import reactor class HTTPGetter(client.HTTPClientFactory): protocol = client.HTTPPageGetter class Fetcher: def __init__(self,client_factory = HTTPGetter): self.factory = client_factory def download(self,host,port,url): f = self.factory(url) f.deferred.addCallback(self.downloadFinished).addErrback(self.downloadFailed) k = reactor.connectTCP(host, port, f, timeout=10) return f.deferred def downloadFinished(self,v): print "good" def downloadFailed(self, v): print "bad" print v r = Fetcher() w = r.download("www.shopzilla.com",80,"/aaaa") reactor.callLater(10,reactor.stop) reactor.run() This results in: Unhandled error in Deferred: Traceback (most recent call last): File "/usr/local/lib/python2.4/site-packages/twisted/internet/posixbase.py", line 226, in mainLoop self.runUntilCurrent() File "/usr/local/lib/python2.4/site-packages/twisted/internet/base.py", line 541, in runUntilCurrent call.func(*call.args, **call.kw) File "/usr/local/lib/python2.4/site-packages/twisted/internet/tcp.py", line 494, in resolveAddress d.addCallbacks(self._setRealAddress, self.failIfNotConnected) File "/usr/local/lib/python2.4/site-packages/twisted/internet/defer.py", line 182, in addCallbacks self._runCallbacks() --- <exception caught here> --- File "/usr/local/lib/python2.4/site-packages/twisted/internet/defer.py", line 307, in _runCallbacks self.result = callback(self.result, *args, **kw) File "/usr/local/lib/python2.4/site-packages/twisted/internet/tcp.py", line 498, in _setRealAddress self.doConnect() File "/usr/local/lib/python2.4/site-packages/twisted/internet/tcp.py", line 520, in doConnect connectResult = self.socket.connect_ex(self.realAddress) File "<string>", line 1, in connect_ex exceptions.TypeError: an integer is required Which is apparently due to the fact that doConnect assumes a good address and so does not trap for TypeError. The bad address that doConnect blows up on ('',None) for (host,port) slips in due to twisted.web.client.handleStatus_301. The example site (Shopzilla.com) posts a URL for the 301 Location that is not fully qualified. handleStatus_301, in the face of such a URL, appears to fail because it relies on getting the host/port from the location URL, but these are not present in it. Thus it passes in the ('',None) to its reactor.connectTCP attempt to follow the redirect, leading to the error above. My kludge fix to handleStatus_301 is given below, where if the host or port are missing I steal them from the transport, which should be correct since it was just used to get the page. I am running with this now, with no errors. Is this a Twisted issue? If so, is my fix reasonable? If it is not a Twisted issue, what am I doing wrong? Thanks, Keith def handleStatus_301(self): l = self.headers.get('location') if not l: self.handleStatusDefault() url = l[0] if self.followRedirect: scheme, host, port, path = \ _parse(url, defaultPort=self.transport.getPeer().port) self.factory.setURL(url) #following 4 lines added kad to fix apparent issue with 301 to a url that is not fully qualified if self.factory.host == '': self.factory.host = self.transport.addr[0] if self.factory.port == None: self.factory.port = self.transport.addr[1]

2 1