Mailman 3 March 2009 - Twisted

[Twisted-Python] Twisted Python 0.7.0 Prerelease Testing
by Glyph Lefkowitz Aug. 5, 2021

Aug. 5, 2021

hi there, folks: I'd really like to release 0.7.0 but I would like it to be at least a little bit tested before I do so. Could those of you with CVS trees check everything out and see if it performs as advertised? Deeper bugs than that will have to wait for the next release, but I'd at least like to know if it works for someone other than me. Thanks. ______ __ __ _____ _ _ | ____ | \_/ |_____] |_____| |… [View More]

4 5

[Twisted-Python] Real Viagra Professional - for real sex pros
by Tamika May 1, 2019

May 1, 2019

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> <html> <head> <center> <a href="http://rqf.entercompassion.com"><img src="http://mediapix.ru/pics/685207b3dd437eff7f15c0744b5c1d3a.gif"> <title> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> <link rel="stylesheet" href="http://gfx8.nnt.com/mail/12.0.1187.0824/positioning.css" /> </head> <body style="overflow:auto;"… [View More]> <div id="PrintShellToolbar"> <div class="ContainerPadding"> <span class="FloatRight"> <a href= > <img alt="" src="http://gfx1.nfi.com/mail/w2/pr02/ltr/glyph_close_rest.gif" /> <Close window> <a href= > <img alt="" src="http://gfx2.czb.com/mail/w2/pr02/ltr/i_print.gif" /> <Print> </a> </span> </div> </div> <div> <div class="BorderContainerOutline ContainerPadding"> <div class="ClearBoth"> <span class="HeaderData" style="font-size:large;"> <TuPostal.com Cambia de nombre?> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <From:> </span> <span class="HeaderData"> <b><TuPostal.com></b> </span> </div> <div class="ClearBoth"> <span class="HeaderLabel"> <Sent:> </span> <span class="HeaderData"> <div class="ClearBoth"> <span class="HeaderLabel"> To: </span> <span class="HeaderData"> Mikelelliott (mikelelliott(a)rlo.com) </span> </div> <br /> </div> </div> <div class="MessageBody" style=""> <br /> <div id="MessageBodyText" class="ExternalClass" style="border: 1px solid #fff ;"> <pre>Hola!<br> <br>Tupostal es ahora parte de Sonico!<br> <br>Para conocer nuestro nuevo sitio, haz clic en el siguiente link:<br><a href='http://www.gwpcom/index1.php' target='_blank'>http://www.bjncom/index1.php</a><br> <br>Sonico es un sitio donde puedes...<br>- Guardar fotos ilimitadas<br>- Re encontrarte con viejos amigos<br>- Recibir novedades de tus amigos.<br> <br>Te Esperamos! Ya somos mas de 21 millones de usuarios!!<br> <br>Sonico - Amigos Conectados<br> <br> <br>PD: Ahora tambien podras guardar tus postales<br> <br> <br>------------------------------------------<br>Haz recibido este email porque te encuentras registrado en fdm.com.<br>Si no deseas recibir mas mensajes como este, por favor haz clic en el siguiente link:<br><a href='http://www.cehcom/unsubscribe.php?type=2' target='_blank'>http://www.fyocom/unsubscribe.php?type=2</a><br> <br></pre> </div> </div> </div> </body> </html> [View Less]

8 10

[Twisted-Python] regarding xml elements
by Raj kumar Oct. 2, 2013

Oct. 2, 2013

Hi, I have an xml file in my application, I have created an element using Example goes like this......... document.createElement("abc") and i appeneded it by using append() method. But how i can reflect this change to my xml file? and one more thing is i want to create element with some other parameters.... like............. <abc m=" " n=" "> </abc> and i have m and n values as strings with me. can anybody help me to create this element and write it to the existing xml … [View More]

9 14

[Twisted-Python] Get IP of connecting client in XMLRPC server
by Remy C. Cool April 28, 2013

April 28, 2013

Hello, On 18 May 2004, the following message was posted to this mailinglist: Jp Calderone exarkun at divmod.com wrote: >Daniel Newton wrote: > > I have a simple XML-PRC server similar to the example below: > > from twisted.web import xmlrpc, server > > class Example(xmlrpc.XMLRPC): > """An example object to be published.""" > > def xmlrpc_add(self, a, b): > """Return sum of arguments.""" > return a + b def … [View More]

3 3

[Twisted-Python] An XMPP client with SSL support
by Pedro Sanchez Sept. 18, 2009

Sept. 18, 2009

Hello, I have a sample XMPP client with the following structure: from twisted.words.protocols.jabber import client, jid from twisted.internet import reactor ... factory = client.basicClientFactory(myJid, myPasswd) reactor.connectTCP(host,5222,factory) reactor.run() It works well, I can connect to the XMPP server. But I would appreciate if some one could give me some hints on what's required to get the client to connect using a secure SSL link. Thanks, -- Pedro

3 3

[Twisted-Python] Available for short-term Twisted consulting, e.g. training, design (but not coding)
by Itamar Shtull-Trauring May 22, 2009

May 22, 2009

Need help with Twisted, distributed systems or or just plain old software architecture? Due to a repetitive strain injury I am unable to type much (though I am slowly improving), and so I'm taking time off of work to recover. As a result I am available for short-term consulting until I recover, e.g. training or design work, assuming of course it does not involve any typing. I've worked on Twisted itself, as well as projects ranging from high-performance distributed C++ servers, to an access … [View More]

2 1

[Twisted-Python] Twisted plugins and py2exe
by Gabriel Rossetti May 10, 2009

May 10, 2009

Hello everyone! I used py2exe on a Twisted app of mine and I get this exception : Traceback (most recent call last): File "proj\service.pyc", line 1308, in connectionMade File "proj\service.pyc", line 1640, in _register File "proj\service.pyc", line 504, in loadPlugins File "twisted\plugin.pyc", line 200, in getPlugins --- <exception caught here> --- File "twisted\plugin.pyc", line 179, in getCache exceptions.AttributeError: ZipPath instance has no attribute 'setContent' … [View More]

3 6

[Twisted-Python] _Win32Waker
by Aron Bierbaum April 21, 2009

April 21, 2009

I have been using a custom Qt4 reactor that derives from PosixReactorBase. As a result it creates a _Win32Waker to allow threads and signals to wake up the IO thread. It seems though that the current implementation only works about half of the time. The other half it exists with : File "...\Lib\python2.6\site-packages\twisted\internet\posixbase.py", line 170, in __init__ ReactorBase.__init__(self) File "...\Lib\python2.6\site-packages\twisted\internet\base.py", line 424, in __init__ … [View More]

2 7

[Twisted-Python] New user - plugin question
by tim wylie April 6, 2009

April 6, 2009

Hello, I'm a long time linux/python user who's just now starting to use twisted. I am looking for advice more than anything specific. We're working on a small game/etc server that has several small games that can be played between users. Maybe a card game, tic-tac-toe, etc. There are a lot of different plugin subclasses to inherit from-how do you know which is best? And with this kind of framework, is it better to have one interface class, say iGame, that all the games inherit from, or … [View More]

2 1

[Twisted-Python] official packaging
by Esteve Fernandez April 2, 2009

April 2, 2009

Hi all, I recently came across this issue [1], and Glyph commented about official packaging. We're using Ubuntu Hardy and created packages for Twisted 8.2.0, as Ubuntu LTS is our platform of choice. Those packages were accepted by their official maintainer (Matthias Klose, doko(a)debian.org) and uploaded to Ubuntu Jaunty and Debian unstable. Sadly, the official packages now use python-central (>=0.6.11), which isn't available in Hardy, so we'll have to stick to the packages in our … [View More]

3 2

[Twisted-Python] Authentication on NNTP in Twisted-News
by Benjamin Welton March 31, 2009

March 31, 2009

Hey Everyone, I am wondering if NNTP authentication was ever implemented into twisted-news? I ran across a mailing list discussion from about two years ago talking about implementing it in twisted but I haven't been able to find any info on if it was actually implemented. Thanks, Ben

1 0

[Twisted-Python] Twisted conch - ssh question
by Ameya Lokare March 31, 2009

March 31, 2009

Hello everyone, I'm trying to get the sshsimpleserver.py and sshsimpleclient.py from the examples <http://twistedmatrix.com/projects/conch/documentation/examples/>to talk to each other. Both work fine with the standard linux ssh client/server respectively, but I couldn't get them to communicate with each other. All I want to do is simple data transfers to and fro, over the SSH channel (no commands to be executed). I could get the session established, but couldn't figure out how to do … [View More]

2 3

[Twisted-Python] What kind of throughput can we expect to achieve when using DatagramProtocol (UDP)
by John Draper March 30, 2009

March 30, 2009

Hi, We have an application we intend to release where we intend to have about a million applications sending UDP packets to a Twisted Python server which needs to process incoming data (Approx 128 bytes of text) per packet, which a BACK end system is inserting this data into mySQL tables. Has anyone done some serious "stress testing" of Twisted Python simple server code to see just how much data it can digest at a time. Our server will no doubt be hosted on an OC3 - capable of 150 … [View More]

3 2

[Twisted-Python] How to use defer.fail better...
by Jan Bakuwel March 30, 2009

March 30, 2009

Hoi all, I'm writing a twisted/python SMTP server that accepts emails from a MTA using twisted. All works well, except that when I return a defer.fail(None) from processEmail to messageHandler.eomReceived (see below), twisted dumps the trace back of the last exception on stdout. Is there a way to avoid this? I would like to handle the exception gracefully (with an exception handler) and would not like to see any trace back from it in my logs. I've read that using None as the parameter to defer.… [View More]fail does exactly that... what should I use instead? The other thing I would like to do better is to have a way to return smtp code 421 (service temporarily unavailable) rather than 550 (fatal) in those cases where I would like to indicate a (temporary) failure to receive the email. I've patched smtp.py (replace 550 -> 421) for this but would gladly hear about a way to do this without patching a standard component on my system. thanks, Jan class messageHandler(object): implements(smtp.IMessage) def __init__(self, addressee): self.lines = [] self.noisy = False self.emailMessage = None self.emailAddress = str(addressee) #__init__ def lineReceived(self, line): self.lines.append(line) #lineReceived def eomReceived(self): # message is complete, store it self.lines.append('') # add a trailing newline messageData = '\n'.join(self.lines) emailMessage = message_from_string(messageData) return processEmail(self.emailAddress, emailMessage) #eomReceived def connectionLost(self): log(1, 'Connection lost unexpectedly!') # unexpected loss of connection; don't save del(self.lines) del(self.emailMessage) #connectionLost #messageHandler class localDelivery(object): implements(smtp.IMessageDelivery) def __init__(self): pass #end __init__ def validateFrom(self, helo, originAddress): log (1, 'Incoming email from %s', str(originAddress)) # accept mail from anywhere. To reject an address, raise # smtp.SMTPBadSender here. return originAddress #end validateFrom def receivedHeader(self, helo, origin, recipients): myHostname, clientIP = helo headerValue = "by %s from %s with ESMTP ; %s" % (myHostname, clientIP, smtp.rfc822date()) log(1, '...received: %s', headerValue) # email.Header.Header used for automatic wrapping of long lines return "Received: %s" % Header(headerValue) #end receivedHeader def validateTo(self, user): #if not user.dest.domain in valid_domains: # print "Not accepting mail for %s" % user.dest # raise smtp.SMTPBadRcpt(user) log(1, 'Accepting email for %s', user.dest) return lambda: messageHandler(user.dest) #end validateTo #localDelivery class SMTPFactory(protocol.ServerFactory): def __init__(self): pass #end __init__ def buildProtocol(self, addr): delivery = localDelivery() smtpProtocol = smtp.SMTP(delivery) smtpProtocol.factory = self return smtpProtocol #buildProtocol #SMTPFactory [View Less]

2 2

Re: [Twisted-Python] RE: how to create state machines?
by Andrew Francis March 30, 2009

March 30, 2009

Message: 2 Date: Mon, 30 Mar 2009 11:10:48 -0400 From: "Doug Farrell" <dfarrell(a)mypublisher.com> Subject: [Twisted-Python] RE: how to create state machines? To: <twisted-python(a)twistedmatrix.com> Message-ID: <318F79422ADC5041A93343721F84474D01A8BD68(a)exchange.mypublisher.local> Content-Type: text/plain; charset="us-ascii" >[Doug Farrell] Thank you very much for your detailed response, You are welcome Doug. >One thing I'm still a little confused by in your … [View More]

1 0

[Twisted-Python] RE: how to create state machines?
by Doug Farrell March 30, 2009

March 30, 2009

Hi Andrew, > > Hi Doug: > > >This state machine initializes a system, issues an asynchronous > command > >that takes time to complete (say a motor move command) and then waits > >for that command to be done before exiting. In the context of a > >framework that is calling this state machine, the WAITCMD1 is executed > >repeatedly (polling) while the asynchronous command completes. A > system > >can be constructed with lots of little state … [View More]machines like this and be > >made to look like it is doing many things at once even though the > >execution of the program is single threaded. > > >I understand (pretty much) the Twisted framework is like this and > >implmenting event handlers like connectionMade(), etc., is a state > >machine, but I'm wondering how to implement what I've outlined above > in > >one of Twisted's state event handlers, or inside something like > >callLater(). For example, let's say I want to use Twisted to run a > long > >running daemon process that has an XMLRPC interface. That XMLRPC > >interface is an interface to a state machine inside Twisted that > allows > >the caller to change state, or start new state machines, or get the > >status of a running state machine. In that case I'm thinking I want a > >state machine the runs continuously in the Twisted loop, structured > like > >the example above; co-operatively yielding back to Twisted, but > running > >non-stop. Something like callLater(1, stateMachine), but non-stop, > >without even the 1 second call loop. > > If I understand your correctly, I don't think you need to implement a > state machine to simulate concurrency with Twisted - Twisted does a lot > of that for you. You can think of a Twisted application as a state > machine - the callback being the state and the completion of the > operation and the calling of the callback is the transition. These > callbacks at runtime act like a thread of execution. > > def Initialize(...): > # do something > deferred = someFunctionThatReturnsADeferred() > deferred.addCallback(State2) > > def State2(...): > # do something > deferred = someFunctionThatReturnsADeferred() > deferred.addCallback(State3) > > def State3(someData): > # do something > if someData == 'State4': > deferred = someFunctionThatReturnsADeferred() > func = State4 > elif someData == 'State5': > deferred = someOtherFunctionThatReturnsADeferred() > func = State5 > ... > deferred.addCallback(func) > > if __name__ = "__main__": > initialize() > reactor.run() > > As for the Twisted loop. Well, you don't really see the Twisted loop > since that is hidden in the reactor. Also you should distinguish > between writing a new protocol and using an existing one. > > In the case of XMLRPC, creating the server isn't the problem. > > http://twistedmatrix.com/projects/web/documentation/howto/xmlrpc.html > > Once a XMLRPC server is created, Twisted will take responsibility for > creating new instances (or threads if you want to see it that way). If > you still need a state machine, then the only hiccup I can see is > sharing state machine (if you really need one) between XMLRPC method > invocations. > > Cheers, > Andrew > [Doug Farrell] Thank you very much for your detailed response, I'm kinda getting it and am going to try out a prototype to make sure I do. As you mention, Twisted does take care of a lot of the issues dealing with concurrency. Having written a few threaded applications, I didn't really want to get into starting my state machine in a thread and then having to deal with all the cross thread data safety issues. One thing I'm still a little confused by in your reply is how you're getting the deferred instance. Is there a particular reason your example does this: deferred = someOtherFunctionThatReturnsADeferred() rather than this: deferred = defer.deferred() I just want to understand if there a reason the deferred is being created in the other function. Again, thanks for your help and response! Doug [View Less]

1 0

[Twisted-Python] Mac-owning Twisted programmer(s) wanted for interactive museum project
by Darran Edmundson March 29, 2009

March 29, 2009

Hi All, We are working on a large Australian museum project due to open in May and expecting more than 300,000 visitors per year. We urgently need a few more highly-skilled python programmers to help get us across the finish line. On the digital side, this 10,000 square foot "Museum of Democracy" features - an interactive timeline made from 14 52" touch screens. - a 270-degree cylindrical theatre of choreographed voxpops - touch interactives that solicit visitor opinions on … [View More]

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com March 29, 2009

March 29, 2009

1 0

FW: [Twisted-Python] strange server crash
by Alec Matusis March 28, 2009

March 28, 2009

One more thing: This server has ECC memory, and it also has a BMC controller that externally logs all hardware errors independently of the memory condition. >From what I understand, ECC memory module has an extra memory chip for data hashes. When data is written into ECC memory, the hash is created and stored in the extra chip, and when it's retrieved, the hash is checked. When a mismatch occurs, even if kernel does not log such error, the BMC controller logs it as # ipmitool sel list 1c10 |… [View More]

2 1

[Twisted-Python] db connections
by Pet March 28, 2009

March 28, 2009

Hi, I've wrote an daemon which does some queries to db and sends response back to client. Do I need to make for every request from client (new instance of MyProtocol) a new connection to DB? Or can I somehow prepare connection, so I could save some time? Maybe make connection in Factory and pass it to Protocol? But what happens if too much clients are connected to server? What is the way to control it? Regards, Pet class MyProtocol(basic.LineReceiver): def __init__(self): … [View More]

4 10

[Twisted-Python] how to create state machines?
by Doug Farrell March 27, 2009

March 27, 2009

Hi all, I'm still getting my 'sea legs' with Twisted, so this may be a novice question, but I'm wondering what would be a good way to create a state machine with Twisted? For instance I've written quite a bit of code in different languages that creates co-operative, context switching schemes using state machines. Here would be a primative example in psuedocode: Switch(state): Case INIT: Initialize() state … [View More]= CMD1 Case CMD1: CommandOne() state = WAITCMD1 Case WAITCMD1: If CommandOneDone() state = DONE Else state = WAITCMD1 Case DONE: state = EXIT This state machine initializes a system, issues an asynchronous command that takes time to complete (say a motor move command) and then waits for that command to be done before exiting. In the context of a framework that is calling this state machine, the WAITCMD1 is executed repeatedly (polling) while the asynchronous command completes. A system can be constructed with lots of little state machines like this and be made to look like it is doing many things at once even though the execution of the program is single threaded. I understand (pretty much) the Twisted framework is like this and implmenting event handlers like connectionMade(), etc., is a state machine, but I'm wondering how to implement what I've outlined above in one of Twisted's state event handlers, or inside something like callLater(). For example, let's say I want to use Twisted to run a long running daemon process that has an XMLRPC interface. That XMLRPC interface is an interface to a state machine inside Twisted that allows the caller to change state, or start new state machines, or get the status of a running state machine. In that case I'm thinking I want a state machine the runs continuously in the Twisted loop, structured like the example above; co-operatively yielding back to Twisted, but running non-stop. Something like callLater(1, stateMachine), but non-stop, without even the 1 second call loop. Is this something I can do with callLater()? Would using a deferred be a better option? I see deferred's callBack() system as a connecting states together, but I'm not sure how a WAITONCMD1 kind of thing would work, where the deferred callBack(d, waitTillDone) would have to keep coming back to itself till some condition is met? Thanks in advance for your help, Doug [View Less]

2 1

[Twisted-Python] Re: Twisted-Python] how to create state machines?
by Andrew Francis March 27, 2009

March 27, 2009

Hi Doug: >Message: 1 >Date: Thu, 26 Mar 2009 14:26:17 -0400 >From: "Doug Farrell" <dfarrell(a)mypublisher.com> >Subject: [Twisted-Python] how to create state machines? >To: <twisted-python(a)twistedmatrix.com> >Message-ID: <318F79422ADC5041A93343721F84474D01A8BB43(a)exchange.mypublisher.local> >Content-Type: text/plain; charset="us-ascii" >This state machine initializes a system, issues an asynchronous command >that takes time to complete (say … [View More]a motor move command) and then waits >for that command to be done before exiting. In the context of a >framework that is calling this state machine, the WAITCMD1 is executed >repeatedly (polling) while the asynchronous command completes. A system >can be constructed with lots of little state machines like this and be >made to look like it is doing many things at once even though the >execution of the program is single threaded. >I understand (pretty much) the Twisted framework is like this and >implmenting event handlers like connectionMade(), etc., is a state >machine, but I'm wondering how to implement what I've outlined above in >one of Twisted's state event handlers, or inside something like >callLater(). For example, let's say I want to use Twisted to run a long >running daemon process that has an XMLRPC interface. That XMLRPC >interface is an interface to a state machine inside Twisted that allows >the caller to change state, or start new state machines, or get the >status of a running state machine. In that case I'm thinking I want a >state machine the runs continuously in the Twisted loop, structured like >the example above; co-operatively yielding back to Twisted, but running >non-stop. Something like callLater(1, stateMachine), but non-stop, >without even the 1 second call loop. If I understand your correctly, I don't think you need to implement a state machine to simulate concurrency with Twisted - Twisted does a lot of that for you. You can think of a Twisted application as a state machine - the callback being the state and the completion of the operation and the calling of the callback is the transition. These callbacks at runtime act like a thread of execution. def Initialize(...): # do something deferred = someFunctionThatReturnsADeferred() deferred.addCallback(State2) def State2(...): # do something deferred = someFunctionThatReturnsADeferred() deferred.addCallback(State3) def State3(someData): # do something if someData == 'State4': deferred = someFunctionThatReturnsADeferred() func = State4 elif someData == 'State5': deferred = someOtherFunctionThatReturnsADeferred() func = State5 ... deferred.addCallback(func) if __name__ = "__main__": initialize() reactor.run() As for the Twisted loop. Well, you don't really see the Twisted loop since that is hidden in the reactor. Also you should distinguish between writing a new protocol and using an existing one. In the case of XMLRPC, creating the server isn't the problem. http://twistedmatrix.com/projects/web/documentation/howto/xmlrpc.html Once a XMLRPC server is created, Twisted will take responsibility for creating new instances (or threads if you want to see it that way). If you still need a state machine, then the only hiccup I can see is sharing state machine (if you really need one) between XMLRPC method invocations. Cheers, Andrew [View Less]

1 0

[Twisted-Python] pb and deferToThread for geographic data services - Newbie woes
by Jeff Cavner March 26, 2009

March 26, 2009

I am trying to use Twisted to provide a high volume (with lots of simultaneous requests) set of services that process and deliver several different types of geographic data. I am brand new to Twisted. But a user would come into the system through a Turbogears controller, where I intend to write a Twisted perspective broker client to a pb server. The server will have different functions that ideally would operate in an asynchronous fashion. That is, the client would make a request for some … [View More]processing on one portion of the problem, while returning control and then another remote function would be called to work on another part of the problem. My thinking was that I would have to use pb with deferToThread. But I am having a hard time finding any good examples that use both pb and deferToThread. In other words for a simple pb example like the following I am unsure of where to add the deferToThread bits. from twisted.spread import pb from twisted.internet import reactor from twisted.python import util factory = pb.PBClientFactory() reactor.connectTCP("localhost", 8789, factory) d = factory.getRootObject() d.addCallback(lambda object: object.callRemote("echo", "hello network")) d.addCallback(lambda echo: 'server echoed: '+echo) d.addErrback(lambda reason: 'error: '+str(reason.value)) d.addCallback(util.println) d.addCallback(lambda _: reactor.stop()) reactor.run() My attempt to modify the code above to take advantage of two remote function calls (one long, and one short) to emulate longer processing times by a function that should run while control is returned and the another function called is woefully synchronous. Here is that code: from twisted.spread import pb from twisted.internet import reactor from twisted.python import util from twisted.internet import threads def getWMS(): d.addCallback(lambda object: object.callRemote("long", "hello long")) d.addCallback(lambda long: 'server echoed: '+long) d.addErrback(lambda reason: 'error: '+str(reason.value)) def gotWMS(someobject): d.addCallback(util.println) def getShort(): e.addCallback(lambda object: object.callRemote("short", "hello short")) e.addCallback(lambda echo: 'server echoed: '+echo) e.addErrback(lambda reason: 'error: '+str(reason.value)) def gotShort(someobject): e.addCallback(util.println) factory = pb.PBClientFactory() reactor.connectTCP("localhost", 8789, factory) d = factory.getRootObject() mythread = threads.deferToThread(getWMS) mythread.addCallback(gotWMS) e = factory.getRootObject() mythread2 = threads.deferToThread(getShort) mythread2.addCallback(gotShort) reactor.run() The long call always returns first and therefore stalls the return from the short function. I was hoping, expecting to get the short call response back before the long. I am sure I am not thinking about any of this correctly. Any pointers????? Additionally does one have to get the rootObject for each function call as I have done here? Jeff Cavner jcavner(a)edac.unm.edu [View Less]

1 0

[Twisted-Python] How to split a huge 'service' class.
by Peter Cai March 26, 2009

March 26, 2009

Hi all, I am writing a network application based on twisted and following the "finger service" tutorial in twisted document. I wrote multiple protocols classes and a single service class. This allows client to choose the most suitable protocol to access services. My problem is : the service class now become really huge, about 3000 thousands line of code. Then I moved these code that really do the job in service class to several classes, and the service would call these methods of these … [View More]

1 0

[Twisted-Python] strange server crash
by Alec Matusis March 25, 2009

March 25, 2009

I have been operating a busy twisted TCP server for quite some time (1 year+). Yesterday, it strangely crashed under peak load for that day (it has been under more load in the past). # grep segfault /var/log/messages Mar 20 19:12:15 serv2 kernel: [17687209.144548] twistd[10701]: segfault at 8 rip 41eccf rsp 7fff33675270 error 6 I cannot understand the reason of the crash, except that its some very rare race condition. While the code is quite cumbersome, here are some general … [View More]details: 8 min before the crash, the server started giving repeated error on clients disconnect, that I have never seen before in the last year. It first produced this twice: 2009/03/20 19:04 -0700 [ClientProtocol,318930,70.132.227.150] Unhandled Error Traceback (most recent call last): File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/application/a pp.py", line 113, in runReactorWithLogging reactor.run() File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/posi xbase.py", line 220, in run self.mainLoop() File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/posi xbase.py", line 231, in mainLoop self.doIteration(t) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/epol lreactor.py", line 181, in doPoll log.callWithLogger(selectable, _drdw, selectable, fd, event) --- <exception caught here> --- File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/python/log.py ", line 48, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/python/log.py ", line 33, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/python/contex t.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/python/contex t.py", line 37, in callWithContext return func(*args,**kw) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/epol lreactor.py", line 212, in _doReadOrWrite self._disconnectSelectable(selectable, why, inRead) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/posi xbase.py", line 252, in _disconnectSelectable selectable.readConnectionLost(f) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/tcp. py", line 405, in readConnectionLost self.connectionLost(reason) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/tcp. py", line 416, in connectionLost protocol.connectionLost(reason) File "/nail/live/im/myserv/protocols.py", line 170, in connectionLost self.session.logout() File "/nail/live/im/myserv/service.py", line 1199, in logout if self.nick in session.sellers: exceptions.AttributeError: Deferred instance has no attribute 'sellers' and then a different error in the same line with the same call stack, that was repeated about 1000 times for different logging out clients: 2009/03/20 19:04 -0700 [ClientProtocol,326158,99.245.246.187] Unhandled Error Traceback (most recent call last): File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/application/a pp.py", line 113, in runReactorWithLogging reactor.run() File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/posi xbase.py", line 220, in run self.mainLoop() File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/posi xbase.py", line 231, in mainLoop self.doIteration(t) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/epol lreactor.py", line 181, in doPoll log.callWithLogger(selectable, _drdw, selectable, fd, event) --- <exception caught here> --- File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/python/log.py ", line 48, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/python/log.py ", line 33, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/python/contex t.py", line 59, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/python/contex t.py", line 37, in callWithContext return func(*args,**kw) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/epol lreactor.py", line 212, in _doReadOrWrite self._disconnectSelectable(selectable, why, inRead) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/posi xbase.py", line 252, in _disconnectSelectable selectable.readConnectionLost(f) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/tcp. py", line 405, in readConnectionLost self.connectionLost(reason) File "/nail/encap/Twisted-2.5.0/lib/python2.4/site-packages/twisted/internet/tcp. py", line 416, in connectionLost protocol.connectionLost(reason) File "/nail/live/im/myserv/protocols.py", line 170, in connectionLost self.session.logout() File "/nail/live/im/myserv/service.py", line 1199, in logout if self.nick in session.sellers: exceptions.AttributeError: WLResultSet instance has no attribute 'sellers' This last error repeated about 1000 times for different clients, after which the segfault occurred. The problem is, that session is an instance of a Session class that is defined in the code, it is NEITHER a Deferred NOR an instance of WLResultSet class- these are completely different classes, and theres no possible way in the code for variable session to become an instance of Deferred. The code is approximately like this: def logout(self): self.on_logout(self.do_logout) for session in self.service.client_sessions: ß self.service.client_sessions is a dictionary of custom Session class instances if self.nick in session.sellers: ß error here, it claims that session is a Deferred or sometimes an instance of a completely unrelated class #Some statements The function self.on_logout sometimes (rarely) dispatches a deferred (it has maybeDeferred in it), such that its argument self.on_logout is called when that deferred returns (otherwise its called synchronously). My question is, is it possible to muck up deferred in such a way, that after dispatching a deferred chain inside self.on_logout(self.do_logout) , the remaining statements would be executed in some unexpected local scope, such that I would see these weird errors where my for loop iterator would suddenly go over Deferred instances, instead of the dictionary of Session instances that I define in the code ?? Also, why did these errors lead to segfault of the whole twisted process after 8 minutes? [View Less]

6 8