Mailman 3 November 2006 - Twisted

[Twisted-Python] Strange blocking in twisted + quickfix service
by Robert Parrott Nov. 15, 2006

Nov. 15, 2006

Hi Folks, I'm trying to wrap the quickfix engine (www.quickfix.org) with a twisted service, in order to create a standalone daemon that will respond to RPC-like commands (such as "start", "stop", "request quote", etc.) However, I'm finding some odd blocking issues that I need to resolve. Basically, the twisted service works fine until I start the quickfix engine (which runs, I believe in a separate thread). Beforehand, calls to the twisted protocol class I've created invoke methods in the … [View More]

2 1

[Twisted-Python] Available for Twisted training
by Itamar Shtull-Trauring Nov. 15, 2006

Nov. 15, 2006

Hi, Due to RSI I am taking a month off from work, so I am available for Twisted or Python-related training or other consulting (e.g. design and architecture help) that would not involve typing. Ideally in Boston area, but I might be open to other locations. I've been using and developing core parts of Twisted since 2001. See http://itamarst.org/resume.html - not really updated since 2004, so doesn't cover more recent projects such as reliable multicast system with congestion control. To … [View More]

1 0

[Twisted-Python] Foolscap-0.0.5 released (aka newpb)
by Brian Warner Nov. 15, 2006

Nov. 15, 2006

I've uploaded another release of Foolscap, the next-generation remote-object communication system for Twisted. (actually I've uploaded two, but 0.0.4 had a brown-paper-bag bug and so it never got announced). The new release makes all Referenceable objects "giftable" by default, making it easier to share objects among multiple machines. There is a new "Reconnector" with which you can announce your intention to connect to a given target and have the Tub perform exponential-backoff retry attempts … [View More]whenever the connection is lost. Also, notifyOnDisconnect() now accepts args and kwargs to be passed to the disconnect handler callback. This release also adds support for making debian packages, for sid, sarge, and dapper. Just type 'make debian-sid' (etc) to get an installable .deb package in the parent directory. The full list of user-visible changes is attached below. No wire-level compatibility changes were introduced in this release, but there will be some in the future. Please see http://twistedmatrix.com/trac/wiki/FoolsCap for complete information about Foolscap. The 0.0.5 release tarball is available from http://twistedmatrix.com/~warner/Foolscap/ . share and enjoy, -Brian User visible changes in Foolscap (aka newpb/pb2). * Release 0.0.5 (04 Nov 2006) ** add Tub.setOption, add logRemoteFailures and logLocalFailures These options control whether we log exceptions (to the standard twisted log) that occur on other systems in response to messages that we've sent, and that occur on our system in response to messages that we've received (respectively). These may be useful while developing a distributed application. All such log messages have each line of the stack trace prefixed by REMOTE: or LOCAL: to make it clear where the exception is happening. ** add sarge packaging, improve dependencies for sid and dapper .debs ** fix typo that prevented Reconnector from actually reconnecting * Release 0.0.4 (26 Oct 2006) ** API Changes *** notifyOnDisconnect() takes args/kwargs RemoteReference.notifyOnDisconnect(), which registers a callback to be fired when the connection to this RemoteReference is lost, now accepts args and kwargs to be passed to the callback function. Without this, application code needed to use inner functions or bound methods to close over any additional state you wanted to get into the disconnect handler. notifyOnDisconnect() returns a "marker", an opaque values that should be passed into the corresponding dontNotifyOnDisconnect() function to deregister the callback. (previously dontNotifyOnDisconnect just took the same argument as notifyOnDisconnect). For example: class Foo: def _disconnect(self, who, reason): print "%s left us, because of %s" % (who, reason) def connect(self, url, why): d = self.tub.getReference(url) def _connected(rref): self.rref = rref m = rref.notifyOnDisconnect(self._disconnect, who, reason=why) self.marker = m d.addCallback(_connected) def stop_caring(self): self.rref.dontNotifyOnDisconnect(self.marker) *** Reconnector / Tub.connectTo() There is a new connection API for applications that want to connect to a target and to reconnect to it if/when that connection is lost. This is like ReconnectingClientFactory, but at a higher layer. You give it a URL to connect to, and a callback (plus args/kwargs) that should be called each time a connection is established. Your callback should use notifyOnDisconnect() to find out when it is disconnected. Reconnection attempts use exponential backoff to limit the retry rate, and you can shut off reconnection attempts when you no longer want to maintain a connection. Use it something like this: class Foo: def __init__(self, tub, url): self.tub = tub self.reconnector = tub.connectTo(url, self._connected, "arg") def _connected(self, rref, arg): print "connected" assert arg == "arg" self.rref = rref self.rref.callRemote("hello") self.rref.notifyOnDisconnect(self._disconnected, "blag") def _disconnected(self, blag): print "disconnected" assert blag == "blag" self.rref = None def shutdown(self): self.reconnector.stopConnecting() Code which uses this pattern will see "connected" events strictly interleaved with "disconnected" events (i.e. it will never see two "connected" events in a row, nor two "disconnected" events). The basic idea is that each time your _connected() method is called, it should re-initialize all your state by making method calls to the remote side. When the connection is lost, all that state goes away (since you have no way to know what is happening until you reconnect). ** Behavioral Changes *** All Referenceable object are now implicitly "giftable" In 0.0.3, for a Referenceable to be "giftable" (i.e. useable as the payload of an introduction), two conditions had to be satisfied. #1: the object must be published through a Tub with Tub.registerReference(obj). #2: that Tub must have a location set (with Tub.setLocation). Once those conditions were met, if the object was sent over a wire from this Tub to another one, the recipient of the corresponding RemoteReference could pass it on to a third party. Another side effect of calling registerReference() is that the Tub retains a strongref to the object, keeping it alive (with respect to gc) until either the Tub is shut down or the object is explicitly de-registered with unregisterReference(). Starting in 0.0.4, the first condition has been removed. All objects which pass through a setLocation'ed Tub will be usable as gifts. This makes it much more convenient to use third-party references. Note that the Tub will *not* retain a strongref to these objects (merely a weakref), so such objects might disappear before the recipient has had a chance to claim it. The lifecycle of gifts is a subject of much research. The hope is that, for reasonably punctual recipients, the gift will be kept alive until they claim it. The whole gift/introduction mechanism is likely to change in the near future, so this lifetime issue will be revisited in a later release. ** Build Changes The source tree now has some support for making debian-style packages (for both sid and dapper). 'make debian-sid' and 'make debian-dapper' ought to create a .deb package. [View Less]

2 2

Re : [Twisted-Python] twisted performance and reactor choice
by Stéphane Brault Nov. 14, 2006

Nov. 14, 2006

Thanks Jean-Paul, my mistake ;-) I must confess that if I use twisted (and I must say I'm quite happy with it), I didn't have time to know it that well. Anyway, I was wondering if changing the reactor could help a little (my application runs on a debian sarge 3.1). In which case would the pollreactor be relevant or should I stick with the default one (selectreactor if I read the documentation right). I have a lot of simultaneous deferred at scheduled times (both database access using zalchemy … [View More]and connections to external sites which can take some times). I know this might not be enough to make a decision, but any help would be welcomed. Thanks again, Stéphane ----- Message d'origine ---- De : Jean-Paul Calderone <exarkun(a)divmod.com> À : Twisted general discussion <twisted-python(a)twistedmatrix.com> Envoyé le : Mardi, 14 Novembre 2006, 18h03mn 31s Objet : Re: [Twisted-Python] twisted performance and reactor choice On Tue, 14 Nov 2006 09:14:39 +0000 (GMT), Stéphane Brault <stephane_brault(a)yahoo.fr> wrote: >Hi, > >My application has lots of concurrent connections, both in and out, since >it communicates with outsides sites. It seems that it is sometimes slowed >down, I guess because there are to many simultaneous deffered. I'd like to >know if I could switch reactor (my application is accessed by XML-RPC and >communicates with external applications through XML-RPC, Http requests and >access ftp sites). I also need to know how large can be the threadpool in >order to increase the number of simultaneous deferred. You've gone a bit too fast here. You guessed at the cause of the slowdown. Instead, you need to profile the application and find out what the cause really is. Your question about the size of the threadpool is a bit of a non sequitur. The threadpool, let alone its size, is unrelated to how many Deferreds might exist in your application at once. Unless you are explicitly using the threadpool (or doing a significant number of DNS lookups without using Twisted Names), the threadpool isn't even related to how many concurrent tasks might be in progress in your application. This is because Twisted only uses the threadpool for an extremely limited number of purposes. Basic network traffic isn't one of these purposes. Jean-Paul _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python ___________________________________________________________________________ Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses http://fr.answers.yahoo.com [View Less]

2 1

[Twisted-Python] Re: [Twisted-bugs] Syntax error when upgrading Debian Python 2.4
by Stephen Waterbury Nov. 14, 2006

Nov. 14, 2006

trac(a)twistedmatrix.com wrote: > New submission from LonelyPixel <nospam(a)unclassified.de>: > > I have installed twisted 2.4 for Debian Python 2.4 and after a recent apt-get upgrade, when Python 2.4 was to be upgraded, I get this error: > > Setting up python2.4 (2.4.1-2sarge1) ... > Compiling python modules in /usr/lib/python2.4 ... > Compiling /usr/lib/python2.4/site-packages/twisted/test/stdio_test_halfclose.py ... > File "/usr/lib/python2.4/site-packages/… [View More]

5 8

[Twisted-Python] twisted performance and reactor choice
by Stéphane Brault Nov. 14, 2006

Nov. 14, 2006

Hi, My application has lots of concurrent connections, both in and out, since it communicates with outsides sites. It seems that it is sometimes slowed down, I guess because there are to many simultaneous deffered. I'd like to know if I could switch reactor (my application is accessed by XML-RPC and communicates with external applications through XML-RPC, Http requests and access ftp sites). I also need to know how large can be the threadpool in order to increase the number of simultaneous … [View More]

2 1

[Twisted-Python] advice on asynchronous LDAP?
by Shimon Rura Nov. 14, 2006

Nov. 14, 2006

Hi folks, I'm a newbie writing a twisted server that needs to access an LDAP server. I started by using python-ldap, which worked fine and is well documented, but doesn't seem to have support for a callback mechanism that would make it appropriate for a twisted app. So I switched to ldaptor, which fits the twisted model, but doesn't seem to have much documentation. I'm looking for advice from people who have some familiarity with writing twisted apps that include an LDAP client. Ideally, I'… [View More]

3 2

[Twisted-Python] Why does this simple server not work?
by apocalypznow Nov. 13, 2006

Nov. 13, 2006

The following is run but almost immediately terminates without any errors. WHY? I used "twistd -noy server.tac". Here is the code: **************************** import twisted from twisted.spread import pb from twisted.cred import checkers,portal from twisted.internet import reactor from twisted.python import threadable from twisted.python import usage import threading import socket clients={} class ClientRemoteObject(pb.Avatar): global clients def __init__(self): print "… [View More]

2 1

[Twisted-Python] Converting twistd script to NT service using py2exe
by Alvin Wang Nov. 13, 2006

Nov. 13, 2006

http://twistedmatrix.com/pipermail/twisted-python/2004-January/007021.html http://twistedmatrix.com/pipermail/twisted-python/2004-March/007311.html http://twistedmatrix.com/pipermail/twisted-python/2005-April/010038.html *There seems to be the implication that I need tap2ntsvc from the sandbox. Unfortunately, I can not seem to find it in the sandbox. * Is it out of date? Have the files been moved? Is there another way to accomplish the same objective? thanks -- Alvin

2 1

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com Nov. 12, 2006

Nov. 12, 2006

Bug summary ______________________ Summary for 2006-11-05 through 2006-11-12 Bugs opened: 9 Bugs closed: 3 Total open bugs: 783 (+6) |== Type Changes |== Priority Changes |== Component Changes |Defect: +5 |Highest: +1 |Core: +4 |Enhancement: +1 |High: +1 |Trial: +0 |Normal: +2 |Web: +1 |Low: +1 |Web2: +1 |Lowest:… [View More] +1 New / Reopened Bugs ______________________ ===== Highest ===== [#2223] DeferredSemaphore should be marked "stable" (opened by spiv) enhancement core http://twistedmatrix.com/trac/ticket/2223 [#2225] microdom shouldn't bork on malformed tags when beExtremelyLenient is enabled (opened by jml) (CLOSED, fixed) enhancement web http://twistedmatrix.com/trac/ticket/2225 ===== High ===== [#2224] Web2: Behavior of Request.locateResource vs Resource.locateChild is not clearly defined. (opened by dreid) defect web2 http://twistedmatrix.com/trac/ticket/2224 ===== Normal ===== [#2219] isInIOThread() returns False when used in TestCases (opened by sifu) defect trial http://twistedmatrix.com/trac/ticket/2219 [#2220] [SIP] parsing bugs/limitations and patch (opened by antoine) defect core http://twistedmatrix.com/trac/ticket/2220 [#2222] t.i.task.Clock.advance should set the call.called attribute. (opened by ralphm) defect core http://twistedmatrix.com/trac/ticket/2222 ===== Low ===== [#2221] MotdResource gets stale in finger15.py (opened by gfalcon) (CLOSED, duplicate) defect core http://twistedmatrix.com/trac/ticket/2221 [#2227] sux uses "True" as the default value for an attribute (opened by jml) enhancement web http://twistedmatrix.com/trac/ticket/2227 ===== Lowest ===== [#2226] qt4reactor.stop() not closing connections and quitting python (opened by Tantalum) defect core http://twistedmatrix.com/trac/ticket/2226 Closed Bugs ______________________ ===== Highest ===== [#2225] microdom shouldn't bork on malformed tags when beExtremelyLenient is enabled (opened by jml, closed by jml, fixed) enhancement web http://twistedmatrix.com/trac/ticket/2225 ===== Normal ===== [#2144] Add Trial option to stop test run on the first failure (opened by jml, closed by exarkun, duplicate) enhancement trial http://twistedmatrix.com/trac/ticket/2144 ===== Low ===== [#2221] MotdResource gets stale in finger15.py (opened by gfalcon, closed by jerub, duplicate) defect core http://twistedmatrix.com/trac/ticket/2221 [View Less]

1 0