Mailman 3 June 2005 - Twisted

RE: [Twisted-Python] Documentation: producing standalone win32 exesfortwisted
by Henning.Ramm＠mediapro-gmbh.de 24 Jun '05

24 Jun '05

> There are a number of caveats to creating services > (both on the twisted side and the windows side) > which I will make sure to document. That's good; I don't trust in my abilities so far to implement such without advice. http://www.adelux.fr/libre/howto/deployTwistedWindows looks good so far! (Even if I just have no time to try it.) Thank you in advance! Best regards, Henning Hraban Ramm Südkurier Medienhaus / MediaPro Support/Admin/Development Dept.

1 0

[Twisted-Python] Distinguishing TCP connections
by Grant McDonald 24 Jun '05

24 Jun '05

Eric, I came up with a nice way of managing connections on the last project I worked on: The DataConnection is the protocol instance (in this case inheriting from Int32StringReceiver). Each time a protocol connects it calls the registerSource class DataConnection(Int32StringReceiver): def __init__(self, manager): self.manager = manager def closeConnection(self): self.transport.loseConnection() def connectionMade(self): print 'registering connection' self.registerConnection() def connectionLost(self, reason): print 'connection lost, removing....' self.removeConnection() . . [ code omitted for brevity ] . . def registerConnection(self): """ register data connection with connection manager. """ self.manager.registerSource(self) def removeConnection(self): """ remove this connection from the connection manager. """ self.manager.removeSource(self.id) def setId(self, id): self.id = id And the ConnectionManager is the factory class (inherited from the generic Factory class). class ConnectionManager(Factory): def __init__(self, eventhandler): self.ids = 0 self.sources = {} self.eventhandler = eventhandler # -- see comments later def buildProtocol(self, addr): return DataConnection(self) def newId(self): """ get an id that is unique for the life of the connection manager. """ id = self.ids+1 self.ids = id return id def registerSource(self, source): """ register a new source, this assigns the source an id and adds it to the sources dictionary. """ id = self.newId() self.sources[id] = source # set source id source.setId(id) def removeSource(self, id): """ remove the connection indicated by the source id from the current connection pool any subsequent behaviour must be handled in the event manager. """ if self.sources.has_key(id): del self.sources[id] def closeConnections(self): for source in self.sources.values(): source.closeConnection() [ code omitted for brevity ] As you can see the when a connection is made it is registered with the factory instance. This uniquely identifies the connection, and allows you to identify when you get a message which connection it is from. This also simplifies sending a message back along the same channel: To do this you can create an event handler that implements some form of event processing: class EventHandler: def __init__(self, manager): self.manager = manager def messageReceived(self, id, message): # process message ... # create new message newMessage = "this is a test" # send new message back along same connection self.manager.sendMessage(id, newMessage) And in the ConnectionManager add something like the following methods: def propagateMessage(self, id, message): """ This method simply passes the received message through to the EventHandler class instance along with the id of the connection the message was received from. """ self.eventhandler.messageReceived(id, message) def sendMessage(self, id, message): """ This method retrieves the correct connection and sends a message through it. """ if self.sources.has_key(id): self.sources[id].emitMessage(message) And the DataConnection: def stringReceived(self, data): self.manager.propagateEvent(self.id, data) def emitMessage(self, message): """ send a message along the data connection. """ self.sendString(message) If any of this is unclear please let me know. Regards, Grant McDonald ----Original Message----- From: twisted-python-bounces(a)twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com]On Behalf Of Eric Hsu Sent: 21 June 2005 17:33 To: radix(a)twistedmatrix.com Cc: Twisted general discussion Subject: Re: [Twisted-Python] How could I distinguish each TCP connection? Thank you very much! You've been a GREAT help :D 2005/6/21, Christopher Armstrong <radeex(a)gmail.com>: I suggest, instead of using the stock "Factory", making a subclass of it with buildProtocol implemented like so: def buildProtocol(self, addr): p = Factory.buildProtocol(self, addr) self.connections.append (p) return p Or similar (you'll have to define the 'connections' list yourself in __init__). In the protocol's connectionLost, you could then remove the protocol from that list.

4 4

[Twisted-Python] Twisted / NNTP / Mailman
by Jordan Hayes 24 Jun '05

24 Jun '05

Has anyone worked on this integration? I see it as an NNTP server using the Mailman database as the backing-store. Read-only would be fine, actually. /jordan

1 0

[Twisted-Python] PB SturdyRef issue
by Bruce Mitchener 24 Jun '05

24 Jun '05

I'm using something similar in style to a pb.SturdyRef and have run across a problem with it: While it handles pb.DeadReferenceError, it doesn't handle the pb.PBConnectionLost that can come back as an errback from the callRemote. Is there some way that I can juggle deferreds so that the caller can get the response from the server after I reconnect and re-try like what happens in the case of pb.DeadReferenceError? It doesn't seem like I can just attach an errback to the deferred and in that errback, do a failure.trap(pb.PBConnectionLost) and then do something to bind a new deferred into the callback chain ... Is there something that can happen magically by returning a new deferred from the errback that successfully traps the PBConnectionLost? Thoughts? - Bruce

2 1

RE: [Twisted-Python] Documentation: producing standalone win32 exes fortwisted
by Grant McDonald 24 Jun '05

24 Jun '05

All, > In addition to Mary's general 'yes', I'd be especially interested in > "howto make a Win service from a twisted app" (I'd like to run my server > on a server ;-) without logged in user). There are a number of caveats to creating services (both on the twisted side and the windows side) which I will make sure to document. I have created an issue in the bug tracker for the documentation: http://twistedmatrix.com/bugs/issue1092 Regards, Grant M

1 0

RE: [Twisted-Python] Documentation: producing standalone win32 exes fortwisted
by Henning.Ramm＠mediapro-gmbh.de 24 Jun '05

24 Jun '05

> As a general question, would anyone find it useful to have in the twisted documentation specific information on producing twisted win32 executables (be this twistd, a service, or utilising a plugin) using py2exe? I would be happy to create such doco since I have had to produce several different types of win32 exes using twisted components and am aware of some of the pitfalls ;) In addition to Mary's general 'yes', I'd be especially interested in "howto make a Win service from a twisted app" (I'd like to run my server on a server ;-) without logged in user). Best regards, Henning Hraban Ramm Südkurier Medienhaus / MediaPro Support/Admin/Development Dept.

1 0

[Twisted-Python] Documentation: producing standalone win32 exes for twisted
by Grant McDonald 24 Jun '05

24 Jun '05

All, As a general question, would anyone find it useful to have in the twisted documentation specific information on producing twisted win32 executables (be this twistd, a service, or utilising a plugin) using py2exe? I would be happy to create such doco since I have had to produce several different types of win32 exes using twisted components and am aware of some of the pitfalls ;) Grant P.S. How does the new plugin system work with regards to running the plugin? Is it any different? -----Original Message----- From: twisted-python-bounces(a)twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com]On Behalf Of Jp Calderone Sent: 24 June 2005 07:24 To: Twisted general discussion Subject: Re: [Twisted-Python] twisted.plugin (revised) documentation? On Thu, 23 Jun 2005 14:43:38 -0400, Pedro Sanchez <psanchez(a)nortel.com> wrote: >Hello, > >Where can I find documentation on the current twisted.plugin framework? > >The document "Writing a New Plug-In for mktap" at >http://twistedmatrix.com/projects/core/documentation/howto/plugin.html >seems to have outdated information. When I run the mktap command, as >instructed there, this is what I get: > >$ mktap qotd >TwistedQuotes/plugins.tml:7: DeprecationWarning: The >twisted.python.plugin system is deprecated. See twisted.plugin for the >revised edition. tapname="qotd") > >I'm running twisted 2.0.1 and I can't find any new documentation. The >document above is also found in doc/core/howto/plugin.html of the >current twisted distribution. > It's on my list of things to write. In brief, the way plugins are implemented now is: 1) in a module in the twisted.plugins (plural) package. Such modules should be named in such a way so as not to conflict with plugins provided by other parties (eg, Twisted uses the convention of "twisted_foo" for plugins it itself offers). Installers should place plugin modules in the twisted/plugins/ directory and call list(twisted.plugin.getPlugin(twisted.plugin.IPlugin)) afterwards to regenerate the system cache (not doing so is only detrimental in that operation without the cache is much slower). Uninstallers should do something similar to avoid leaving stale cache entries (these will be found and pruned, but lead to at least one ugly traceback, and for a system-wide install, the user probably will not have access to rewrite the cache with bogus entries removed). As a convenience to developers, plugins placed in a _directory_ (not a package) named "twisted/plugins/" in any directory in sys.path will automatically be found by the plugin system as well. This avoids the install process during development and for testing. 2) In these modules, names bound to objects which provide (z.i terminology) twisted.plugin.IPlugin will be considered plugin objects and can potentially be returned by twisted.plugin.getPlugins() 3) Objects providing IPlugin should also provide some other application-specific interface. When calling twisted.plugin.getPlugins(IFoo), only objects providing IFoo will be returned. 4) Plugin "client" code (that is, code calling getPlugin) is then responsible for calling methods appropriate to the interface it passed. For examples of this, see the twisted_ modules in twisted/plugins/. Hope this helps, Jp _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python

2 1

[Twisted-Python] _Dereference problems with jelly, pb.Cacheable, and cyclic references
by Sebastian Hanlon 24 Jun '05

24 Jun '05

I'm having problems trying to spread a data structure using pb.Cacheable. The general framework involves three classes: Alpha, Bravo, and Charlie. Alpha maintains a list of Bravos and Charlies. Bravos can reference Charlies and Charlies reference Bravos. When they get unserialized, though, some of the Charlies have _Dereference objects where references to Bravos should be. The Bravos themselves come across the wire just fine. A simple example program is attached. To run it, first start the server by running 'python jelly-test.py server', then run the client with 'python jelly-test.py client'. You should get a _Dereference in the list of Charlies printed by the client, like such: --cut here-- bravos: Bravo(0) [<__main__.RemoteCharlie instance at 0x00C14DA0>] Bravo(1) [<__main__.RemoteCharlie instance at 0x00C14DA0>, <__main__.RemoteCharlie instance at 0x00C19058>] Bravo(2) [<__main__.RemoteCharlie instance at 0x00C19058>] -------------------------------------------------- charlies: Charlie(Bravo(0),Bravo(1)) Charlie(<twisted.persisted.crefutil._Dereference instance at 0x00C190A8>,Bravo(2)) --cut here-- Can anyone help? I'm running the latest Twisted SVN on Python 2.3.4. -Sebastian #!/usr/bin/python2.3 """ A trivial test problem to illustrate the problems we're having with _Dereference objects cropping up in the wrong places. """ from twisted.spread import pb from twisted.internet import reactor #++++++++++++++++++++++++++++++++++++++++++++++++++# # Here are the data classes #++++++++++++++++++++++++++++++++++++++++++++++++++# class Alpha(pb.Cacheable): def __init__(self): self.b = [] self.c = [] self.observers = [] def getStateToCacheAndObserveFor(self, perspective, observer): self.observers.append(observer) d = self.__dict__.copy() del d['observers'] return d def stoppedObserving(self, perspective, observer): self.observers.remove(observer) def addBravo(self, name): bravo = Bravo(name) self.b.append(bravo) return bravo def addCharlie(self, bravo_one, bravo_two): charlie = Charlie(bravo_one, bravo_two) self.c.append(charlie) return charlie class Bravo (pb.Cacheable): def __init__(self, name): self.name = name # self.connexions = {} self.connexions = [] self.observers = [] def connect(self,other,charlie): # self.connexions[other.name] = charlie self.connexions.append(charlie) def getStateToCacheAndObserveFor(self, perspective, observer): self.observers.append(observer) d = self.__dict__.copy() del d['observers'] return d def stoppedObserving(self, perspective, observer): self.observers.remove(observer) class Charlie (pb.Cacheable): def __init__(self, bravo_one, bravo_two): self.one = bravo_one self.two = bravo_two bravo_one.connect(bravo_two, self) bravo_two.connect(bravo_one, self) self.observers = [] def getStateToCacheAndObserveFor(self, perspective, observer): self.observers.append(observer) d = self.__dict__.copy() del d['observers'] return d def stoppedObserving(self, perspective, observer): self.observers.remove(observer) # remote caches class Cache(pb.RemoteCache): def setCopyableState(self, state): self.__dict__.update(state) class RemoteAlpha (Cache): pass class RemoteBravo (Cache): def __str__(self): return "Bravo(%s)" % (str(self.name)) pass class RemoteCharlie (Cache): def __str__(self): return "Charlie(%s,%s)" % (str(self.one), str(self.two)) pass cacheMap = ( (Alpha, RemoteAlpha), (Bravo, RemoteBravo), (Charlie, RemoteCharlie), ) for x in cacheMap: pb.setUnjellyableForClass(*x) #++++++++++++++++++++++++++++++++++++++++++++++++++# # Here are server classes #++++++++++++++++++++++++++++++++++++++++++++++++++# class AlphaMaker (pb.Root): def __init__(self): self.alpha = Alpha() bravos = [self.alpha.addBravo(x) for x in range(3)] charlies = [self.alpha.addCharlie(bravos[x], bravos[x+1]) for x in range(2)] def remote_getalpha(self): return self.alpha class Server: def run(self): self.port = reactor.listenTCP(12001, pb.PBServerFactory(AlphaMaker())) print "server running..." reactor.run() #++++++++++++++++++++++++++++++++++++++++++++++++++# # Here are client classes #++++++++++++++++++++++++++++++++++++++++++++++++++# class Client: def __init__(self): self.makerRef = None self.alpha = None def run(self): factory = pb.PBClientFactory() reactor.connectTCP("localhost", 12001, factory) factory.getRootObject().addCallbacks(self.gotmaker, self.error) reactor.run() def error(self, obj): print "error:",str(obj) reactor.stop() def gotmaker(self, obj): print "got maker:",obj self.makerRef = obj self.makerRef.callRemote("getalpha").addCallbacks(self.gotalpha, self.error) def gotalpha(self, obj): print "got alpha:",obj self.alpha = obj print "--------------------------------------------------\nbravos:" for b in self.alpha.b: print b, b.connexions print "--------------------------------------------------\ncharlies:" for c in self.alpha.c: print c reactor.stop() #++++++++++++++++++++++++++++++++++++++++++++++++++# # Here is program flow #++++++++++++++++++++++++++++++++++++++++++++++++++# import sys if len(sys.argv) < 2 or sys.argv[1] not in ('client','server'): print >>sys.stderr, "must specify either client or server on command line" sys.exit() if sys.argv[1] == 'client': Client().run() elif sys.argv[1] == 'server': Server().run()

1 0

[Twisted-Python] Capture blocking calls?
by Ian Duggan 24 Jun '05

24 Jun '05

Hi, Does anyone know of any tools that could be used to wrap a python module and capture/report blocking calls made by it? My immediate need is to extend the twisted soap client to support the WSDL proxy that is part of SOAPpy. The current soap proxy looks to be pretty minimal. I'll probably just end up reading through the code this time, but I thought I would throw the idea out there in case anyone had considered this before. Such a tool might make it easier to wrap up/modify libraries and make them usable from Twisted. --Ian

6 9

[Twisted-Python] Accurate delays in callLater
by George Sakkis 24 Jun '05

24 Jun '05

Hello all, I'm using twisted for a realtime simulation and I'm calling reactor.callLater() for scheduling actions to be performed after a fixed short delay (~60msec). However, if there is a cpu-intensive procedure running in the meantime, the timer for the scheduled call may expire much later, an order of magnitude or more if the procedure takes too much time. Is this to be expected in general ? If not, does this may have to with the delay being too short or the cpu-intensive procedure being mainly run in an extension module ? Finally, is there a way to force the required delay (with a tolerance of a few msec) ? Thanks for any insight, George

7 12