Mailman 3 August 2005 - Twisted

[Twisted-Python] Ftp client
by Michele Petrazzo 04 Aug '05

04 Aug '05

I want to port my ftp client class from standard library (ftplib) to twisted ftpclient. I start to work with ftpclient example and with a "standard" ftp server, it work well, of course :), but I'm working on a not standard server (hylafax), that not accept the standard commands, so now with the standard library, every time that I want to communicate with it, I send its internal commands. Is it possible to "translate" my very short and simple code to twisted ftpclint? However, I have already tried to modify the simple generic client example "EchoClient", and it work until I make a "LIST status" request, that it would have to return a short string (100-150 characters), because if I use passive mode (command 'PASV') I receive a 425 error, and if I don't use it, I don't receive anything (I wait...). Can someone point me to the right direction ? Thanks, Michele

2 8

[Twisted-Python] [andrea@cpushare.com: Re: error after launching cpushare client]
by Andrea Arcangeli 04 Aug '05

04 Aug '05

Hello, a CPUShare user reported a failure in the process protocol processEneded method. The status parameter passed to the processEnded callback is like this: status.value.signal == 11 status.value.status == 139 My server code was validating the sigsegv status returned by the client, and it noticed it wasn't 11. But status.value.signal == 11. See the debugging patch that produced the below output: http://www.cpushare.com/hypermail/cpushare-discuss/05/08/0018.html How can it be that the same status has .value.signal == 11 and .value.status == 139 at the same time? I suspect this is a twisted bug. Thanks for any help! ----- Forwarded message from Andrea Arcangeli <andrea(a)cpushare.com> ----- Date: Wed, 3 Aug 2005 16:35:37 +0200 From: Andrea Arcangeli <andrea(a)cpushare.com> To: cpushare-discuss(a)cpushare.com Subject: Re: error after launching cpushare client On Wed, Aug 03, 2005 at 03:26:39PM +0200, Loïc Le Loarer wrote: > Le Wednesday 03 August 2005 à 13:09:35 +0200, Andrea Arcangeli a écrit: > > Hi, > > > > On Mon, Aug 01, 2005 at 06:36:13PM +0200, lll+cpushare(a)m4x.org wrote: > > > 2005/08/01 17:48 CEST [cpushare_protocol,client] 'sigsegv not killed with signal 11: status 139.' > > > > This is weird, it looks like twisted reports status 139 and signal 11 at > > the same time (while status should be 11 == signal), at first glance it > > looks a twisted bug. > > > > Can you please try to apply this patch and see what it prints? (this way > > it will be printed on the client side, previously the number 139 is > > generated by the server side) > > Hi, > > I have applyed your patch and here is the output : > 2005/08/03 15:23 CEST [-] Log opened. > 2005/08/03 15:23 CEST [-] twistd 2.0.1 (/usr/bin/python 2.4.1) starting up > 2005/08/03 15:23 CEST [-] reactor class: twisted.internet.selectreactor.SelectReactor > 2005/08/03 15:23 CEST [-] Loading cpushare.tap... > 2005/08/03 15:23 CEST [-] Loaded. > 2005/08/03 15:23 CEST [-] Starting factory <cpushare.proto.cpushare_factory instance at 0xb78251ac> > 2005/08/03 15:23 CEST [-] Enabling Multithreading. > 2005/08/03 15:23 CEST [cpushare_protocol,client] Starting seccomp task > 2005/08/03 15:23 CEST [-] Seccomp task gracefully killed by seccomp. > 2005/08/03 15:23 CEST [cpushare_protocol,client] Starting seccomp task > 2005/08/03 15:23 CEST [-] Seccomp task gracefully killed by sigsegv, status 139. > 2005/08/03 15:23 CEST [cpushare_protocol,client] 'sigsegv not killed with signal 11: status 139.' > 2005/08/03 15:23 CEST [cpushare_protocol,client] Lost connection. Reason: [Failure instance: Traceback (failure with no frames): twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost. > 2005/08/03 15:23 CEST [cpushare_protocol,client] ] > 2005/08/03 15:23 CEST [cpushare_protocol,client] <twisted.internet.ssl.Connector instance at 0xb783114c> will retry in 2 seconds > 2005/08/03 15:23 CEST [cpushare_protocol,client] Stopping factory <cpushare.proto.cpushare_factory instance at 0xb78251ac> > 2005/08/03 15:23 CEST [-] Main loop terminated. > 2005/08/03 15:23 CEST [-] Server Shut Down. > > The status is 139, does this help ? Yes it helps since it verified that for some reason twisted reports an exit status of 139, which collides with an exist status of 11. (it wasn't a communication error between client and server) I think it's a twisted bug and not a mistake from my part. I'll ask on the twisted lists to be sure. What distro/twisted are you using? In the meantime this will work around it so you can start earning CPUCoins ;) Index: cpushare/proto.py =================================================================== RCS file: /home/andrea/crypto/cvs/cpushare/client/cpushare/cpushare/proto.py,v retrieving revision 1.49 diff -u -p -r1.49 proto.py --- cpushare/proto.py 6 Jul 2005 23:19:02 -0000 1.49 +++ cpushare/proto.py 3 Aug 2005 14:34:34 -0000 @@ -82,7 +82,7 @@ class state_machine(object): self.protocol.sendString(PROTO_SECCOMP_SUCCESS) def end_failure(failure): end_common() - self.protocol.sendString(PROTO_SECCOMP_FAILURE + struct.pack('!i', failure.value.status)) + self.protocol.sendString(PROTO_SECCOMP_FAILURE + struct.pack('!i', failure.value.signal)) def started(result): self.protocol.sendString(PROTO_SECCOMP_RUN) Thanks! ----- End forwarded message -----

2 4

[Twisted-Python] Manually adding a file descriptor to the reactor
by Jason Fritcher 03 Aug '05

03 Aug '05

Hello, I have a C library I've written an extension wrapper around that manages connections to an external server for me. When I create a connection through this library, I get a file descriptor back that I need to monitor for activity. I've been poking through the twisted code to see how I can add these file descriptors to the reactor, but I'm just not seeing how it can be done. My closest guess right now would be to wrap my file descriptor in a FileDescriptor object, but I'm not seeing how to add that to the reactor, or how to tell it what code to call when there is activity on that descriptor. Can anyone give me some insight into what I need to do, or point me at some documentation that would explain what I need to do? For system details, I'll be doing this with Python 2.3.5, Twisted 2.0.1, on both Mac OS X Tiger, and Red Hat Linux. Thanks. -- Jason Fritcher Software Engineer Core Infrastructure Services & Strategy Earthlink, Inc fritcher(a)corp.earthlink.net (404) 748-7262, x22262

1 0

[Twisted-Python] Twisted api docs
by snacktime 03 Aug '05

03 Aug '05

I remember there being very complete api documentation for twisted 1.3, but I can't find anything similar on the website for twisted 2. Do they no longer exist or am I just blind? Chris

3 3

[Twisted-Python] process status
by Daniel Isenegger 02 Aug '05

02 Aug '05

How to query process state and receive back control? Hi, a newbie question: i struggle with the defer-mechanism. i start from a client program a using defers a protocol.ProcessProtocol-class to invoke a local application (grass) to display a raster. This works fine, the problem is now how to propagate back the final state of the invoked process back to the invoking instance and then to get the control back over the shell the process has been running in. I can see that the state of the invoked process can be accesses in processEnded(), but when this function is invoked the connection to the invoking instance is already closed, so the state cannot be propagated to this invoking instance. Additionaly the shell the process has been invoked does not receive further output of the invoking process., so i assume that the control is not handed back to the invoking process Any hints? Thanks Dani In the following some excerpts of code to illustrate my problem: invoking process: ############################################################# # invokes GIS to display the raster cmd, using IpodlasProcess to # call a process on the machine def displayRaster(self): # init deferred object d d = defer.Deferred() # add the pair of callbacks d.addCallback(self.handleDisplayRaster) d.addErrback(self.handleDisplayRasterFailure) # #invoke the deferred functions: i.e. handleDisplayRaster # and handleDisplayRasterFailure d.callback(rasterfile) print "displayRaster: after d.callback()" # ############################################################ # handles the successfull call of the display of the raster theRaster def handleDisplayRaster(self,theRaster): # # init IpodlasProcess ip = IpodlasProcess() # # # invoke IpodlasProcess to invoke Grass to display the raster theRaster ip.displayRaster(theRaster) # ######################################################### # handles the failed call of the display of the file theFoile # and its return object theFailure def handleDisplayRasterFailure(self,theFailure): # theFailure.trap(RuntimeError) # ############################################################################### # IpodlasClientFactory is derived from ReconnectingClientFactory, which tries # to reconnect if connection is lost class IpodlasClientFactory(ReconnectingClientFactory): # creates protocol and receives events relating to connection state # def startedConnecting(self, connector): print 'Started to connect.' def buildProtocol(self, addr): print 'Connected.' print 'Resetting reconnection delay' self.resetDelay() # associated subsystem is the GUI return IpodlasClient() def clientConnectionLost(self, connector, reason): print 'Lost connection. Reason:', reason # tries to reconnect, if conn fails with exponentially delays ReconnectingClientFactory.clientConnectionLost(self, connector, reason) # # if connection could not be established def clientConnectionFailed(self, connector, reason): print 'Connection failed. Reason:', reason ReconnectingClientFactory.clientConnectionFailed(self, connector, reason) ################################################################### if __name__ == "__main__": # if len(sys.argv) > 1: # set global subsystem assoSubsys = sys.argv[1] #print assoSubsys # from twisted.internet import reactor reactor.connectTCP("localhost",51423,IpodlasClientFactory()) reactor.run() invoked Process: #! /usr/bin/python # File:ipodlasProcess.py """ ipodlasProcess.py usage: python ipodlasProcess.py """ from twisted.internet import protocol from twisted.internet import reactor from twisted.internet import utils #needed to query process termination status import re import os import gc # own classes class IpodlasProcess(protocol.ProcessProtocol): def __init__(self): self.data = "" # # def connectionMade(self): print "IP connectionMade" self.transport.closeStdin() # data from standard output is receieve in outReceived. def outReceived(self, data): print "IP outReceived: " + self.data self.data = self.data + data # def errReceived(self, data): print "IP errReceived():" + data # def inConnectionLost(self): print "IP inConnectionLost! stdin is closed! (we probably did it)" # def outConnectionLost(self): print "IP outConnectionLost data: " + self.data # def errConnectionLost(self): print "IP errConnectionLost! stdin is closed! (we probably did it)" # def processEnded(self, status_object): self.processStatus = status_object.value.exitCode print "IP processEnded, status: " + str(status_object.value.exitCode) \ + "\nquitting " # this is outcommented, since otherwise the reactor of ipodlasClient would # stopped, since this is the one which controls ipodlasProcess (the own in # the main fctn is only started, when invoked from cmd-line) # reactor.stop # def displayRaster(self,theRaster): # cmd = "d.rast" arg1 = theRaster arg2 = "bg=white" # processes are runned trough reactor using spawnProcess # invoke process cmd with the args arg1, arg2 and the parent process # environment env reactor.spawnProcess(self,cmd,[cmd,arg1,arg2],env=os.environ) # print "IP displayRaster" ################################################################### if __name__ == "__main__": # ip = IpodlasProcess() ip.displayRaster("upperEngadine_raster") # reactor.run()

1 0

[Twisted-Python] Sydney Twisted Sprint (19-21/8): Linux Australia grants
by Mary Gardiner 02 Aug '05

02 Aug '05

Hi everyone, Another reminder about the Sydney Twisted Sprint on 19th-21st August, but this time a reminder with a difference... Linux Australia is sponsoring the sprint and is providing a few grants. They are providing two subsidised domestic airfares and three accommodation subsidies to needy Twisted folk, and further provided some funding for a sprint dinner. I need to hear from people wanting the airfares or accommodation subsidies by this Friday by email. People wanting to come to the dinner just need to RSVP on the wiki. Info is at these pages: - http://users.puzzling.org/users/mary/TwistedSprint/moin.cgi/SponsoredFlights - http://users.puzzling.org/users/mary/TwistedSprint/moin.cgi/SponsoredAccomm… - http://users.puzzling.org/users/mary/TwistedSprint/moin.cgi/SponsoredDinner General information on the sprint is available at http://users.puzzling.org/users/mary/TwistedSprint/moin.cgi/FrontPage Thanks very much to both Linux Australia and to Matthew Beauregard of UTS's Programming Society (Matthew got the venue together for us). -Mary

1 0

[Twisted-Python] Engineering Opportunities at Google
by Ravi Pindiproli 01 Aug '05

01 Aug '05

Hello Folks My name is Ravi Pindiproli and I am a Recruiter for the SRE (Site Reliability Engineering) Group at Google. This is the most dynamic, challenging and fun group at Google. The SRE group is responsible for our Google website from start to finish and it is different from the Operations group. We are constantly seeking Extraordinary developers and UNIX Administrators (Linux) experts to join our exciting team and growing organization. We have positions open in the USA in Mountain View (CA), Santa Monica (CA), New York (NY), and Kirkland (WA) as well as Dublin (Ireland) and Zurich (Switzerland). I am enclosing job descriptions of two positions. If any of you feel that either of these positions is something that may be of interest to you, please send me an email so we can schedule a time for me to call you to speak with you to discuss more details. Thanks! Senior Systems Administrator/Architect, Google.com (SRE) This position is available in Mountain View, CA, / New York, NY / Kirkland, WA / Dublin, Ireland / and Zurich, Switzerland. Are you a detail-oriented systems professional, seasoned Unix jockey, Internet services architect, database whiz, performance analyst, or software engineer, at the top of your game? * Looking for the next cool problem to solve, in a fast-paced and engineering-driven environment? * Can you write a script or application that performs practically the entire job you once held earlier in your career? * Does your idea of a good time involve taking things apart just to see how they work? Are you successful putting them back together when you're done? * Can you help achieve increased Availability, Reliability, Uptime, and Quality? (for our systems and services, naturally) If you answered "Yes" to more than three of the above questions, you may have what it takes. Keep reading. Google.com Engineers are involved in all aspects of running our site. From design to debug, from architecture to applications, Google.com engineers spend their time troubleshooting problems in large-scale clustered application service environments, and focus on making continual improvements to our services. Working for Google.com, you will wear many hats: performance analyst, service architect, system/database administrator, capacity planner, tools developer, monitoring expert, and technical evangelist, for all components of Google.com's services. This position requires a passion for problem solving, deep understanding of networks, operating systems, multi-tiered Internet services, and the ability to learn new applications and concepts quickly. Strong scripting skills are essential. Requirements: * 3+ years experience in Unix or Linux Systems Administrator. * Experience in a high-volume or critical production service environment. * Strong understanding of networking; understanding of how to isolate, diagnose, and resolve service delivery components (service delivery components include servers, networks, and applications); be familiar with the functionality, operating, and failure modes of key networking devices (routers, switches, bridges, firewalls, hardware load balancers); be able to identify networking as the potential cause of a service issue using server-resident tools to generate this data (tcpdump, ping, traceroute, etc.); familiarity in interpreting the output of these tools; familiarity with common network topologies, protocols, and tools; have some notion of common network security exploits -- and their remedies; understanding of troubleshooting at the packet level. * Skilled in leading projects, especially in deploying live end-user systems. * Proven technical troubleshooting experience. * Database administration experience a plus. * Programming or debugging of web applications running under Java VM. * In-depth knowledge of Linux, networking, and shell scripting. * Knowledge of Python programming a plus. * Experience with MySQL a plus. * Programming and troubleshooting skills in C/C++ and/or Java a plus. * Excellent written and verbal communication skills. * Ability to handle periodic on-call duty as well as out-of-band requests. * BA/BS in Computer Science or a related technical discipline. Senior Software Engineer/Unix System & Network Administrator, Google.com (SRE) This position is available in Mountain View, CA, / New York, NY / Kirkland, WA / Dublin, Ireland / and Zurich, Switzerland. Are you part ace coder, part adrenaline junkie? Do you have a knack for seeing a problem and immediately discerning the likely solution? Maybe you've been coding for years, are bored with the old design-build-review-test-ship-repeat routine, and yearn for some faster-paced challenges? Or perhaps you're a seasoned software engineer who is also a genius at jockeying networks and administering UNIX clusters. We're looking for top-notch thrill seeking, software engineers to join the Google.com team. Google.com engineers are in the thick of everything involved with keeping Google running, from code-level troubleshooting of traffic anomalies to maintenance of our most cutting edge services; from monitoring and alerts to building new automation infrastructure. We are aggressively building this elite team of high level engineers in this mission critical environment. All team members must have strong analytical and troubleshooting skills, fluency in coding, good communication skills, and most of all enthusiasm for tackling the complex problems of scale which are uniquely Google. Google.com engineers tackle challenging, novel situations every day, and work with just about every other engineering and operations team in the process. Qualifications: * Strong programming/scripting skills in any of the following: C, C++, Java, Perl, Python. * Senior Level experience with Unix system administration. * Strong understanding of networking; understanding of how to isolate, diagnose, and resolve service delivery components (service delivery components include servers, networks, and applications); be familiar with the functionality, operating, and failure modes of key networking devices (routers, switches, bridges, firewalls, hardware load balancers); be able to identify networking as the potential cause of a service issue using server-resident tools to generate this data (tcpdump, ping, traceroute, etc.); familiarity in interpreting the output of these tools; familiarity with common network topologies, protocols, and tools; have some notion of common network security exploits -- and their remedies; understanding of troubleshooting at the packet level. * Strong project management skills, especially in deploying live end-user systems. * In-depth knowledge of Unix (preferably Linux), and shell scripting. * Proven technical troubleshooting experience. * Excellent analytic ability, strong communication skills, and a strong sense of urgency. * Ability to handle periodic on-call duty as well as out-of-band requests. * 6-15+ years experience for Senior SWE position, or 3-8 years experience for SWE position. * BA/BS in CS, or equivalent experience. Ravi Pindiproli G O O G L E Technical Sourcer, SRE 1600 Amphitheatre Pkwy, Bldg., 43 Mountain View, CA 94043 650-253-7888: Direct

1 0