Mailman 3 March 2005 - Twisted

RE: [Twisted-Python] Bootstrapping Servers
by Harald.Nyman＠astrazeneca.com March 24, 2005

March 24, 2005

I do something similar with the help of a shell script - roughly like #!/bin/sh touch START_SERVER while [ -f START_SERVER ] do rm START_SERVER thisDir=`pwd | gawk -F '/' '{print $NF}'` twistd --pidfile=${thisDir}.pid -noy che.tac > /dev/null & done echo Server ${thisDir} stopped. -------------------------------------------------------------------- # Close connections, tell clients whether we're stopping or restarting, ...... def _stopServer(self, restart): if restart: f = open('START_SERVER', 'w') f.close() reactor.stop() sys.exit() Harald -----Original Message----- From: Grant McDonald [mailto:gmcdonald@infocomp.com] Sent: torsdag, 24 mars 2005 07:38 To: Twisted general discussion (E-mail) Subject: [Twisted-Python] Bootstrapping Servers Dear All, I have created some code to close all active connections to a server, spawn a new instance of itself and call reactor.stop() to facilitate a remote bootstrap restart of a server. The bootstrapping process takes place through a loader where if passed an option to bootstrap the code pauses for 20 secs to allow the previous process to exit: import sys if sys.argv[1] == "--bootstrap" or sys.argv[1] == "-b": print 'bootstrapping' del sys.argv[1] import time time.sleep(20.0) from twisted.python.runtime import platformType if platformType == "win32": from twisted.scripts.twistw import run else: from twisted.scripts.twistd import run run() However the above code does not allow the server to start since twistd complains that the port is in use: 2005/03/24 17:14 AUS Eastern Daylight Time [-] Log opened. 2005/03/24 17:14 AUS Eastern Daylight Time [-] twistd 1.3.0 (C:\Python23\python.exe 2.3.3) starting up 2005/03/24 17:14 AUS Eastern Daylight Time [-] reactor class: twisted.internet.default.SelectReactor 2005/03/24 17:14 AUS Eastern Daylight Time [-] managers.ConnectionManager starting on 12005 2005/03/24 17:14 AUS Eastern Daylight Time [-] Traceback (most recent call last): 2005/03/24 17:14 AUS Eastern Daylight Time [-] File "D:\python projects\programs\zcfscheduler\loader.py", line 15, in ? 2005/03/24 17:14 AUS Eastern Daylight Time [-] run() <-- snip --> 2005/03/24 17:14 AUS Eastern Daylight Time [-] return getattr(reactor, 'listen'+self.method)(*self.args, **self.kwargs) 2005/03/24 17:14 AUS Eastern Daylight Time [-] File "C:\Python23\lib\site-packages\twisted\internet\default.py", line 283, in listenTCP 2005/03/24 17:14 AUS Eastern Daylight Time [-] p.startListening() 2005/03/24 17:14 AUS Eastern Daylight Time [-] File "C:\Python23\lib\site-packages\twisted\internet\tcp.py", line 594, in startListening 2005/03/24 17:14 AUS Eastern Daylight Time [-] raise CannotListenError, (self.interface, self.port, le) 2005/03/24 17:14 AUS Eastern Daylight Time [-] twisted.internet.error.CannotListenError: Couldn't listen on any:12005: (10048, 'Address already in use'). how can i get the server to stop, restart itself and release the port in the process? Regards, Grant McDonald Important notice: This message is intended for the individual(s) and entity(s) addressed. The information contained in this transmission and any attached, may be confidential and may also be the subject of legal privilege, public interest immunity or legal professional privilege. Any review, retransmission, dissemination or other use of, taking of any action in reliance upon this information by person or entities other than the recipient is prohibited and requires authorization from the sender. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person) you may not copy or deliver this message to anyone. In such cases you should destroy this message and kindly notify the sender by reply email. WARNING: Although Infocomp has taken reasonable precautions so that no viruses are present in this e-mail, the company cannot accept responsibility for any loss or damage arising from the use of e-mail attachments.

1 0

[Twisted-Python] Bootstrapping Servers
by Grant McDonald March 24, 2005

March 24, 2005

Dear All, I have created some code to close all active connections to a server, spawn a new instance of itself and call reactor.stop() to facilitate a remote bootstrap restart of a server. The bootstrapping process takes place through a loader where if passed an option to bootstrap the code pauses for 20 secs to allow the previous process to exit: import sys if sys.argv[1] == "--bootstrap" or sys.argv[1] == "-b": print 'bootstrapping' del sys.argv[1] import time time.sleep(20.0) from twisted.python.runtime import platformType if platformType == "win32": from twisted.scripts.twistw import run else: from twisted.scripts.twistd import run run() However the above code does not allow the server to start since twistd complains that the port is in use: 2005/03/24 17:14 AUS Eastern Daylight Time [-] Log opened. 2005/03/24 17:14 AUS Eastern Daylight Time [-] twistd 1.3.0 (C:\Python23\python.exe 2.3.3) starting up 2005/03/24 17:14 AUS Eastern Daylight Time [-] reactor class: twisted.internet.default.SelectReactor 2005/03/24 17:14 AUS Eastern Daylight Time [-] managers.ConnectionManager starting on 12005 2005/03/24 17:14 AUS Eastern Daylight Time [-] Traceback (most recent call last): 2005/03/24 17:14 AUS Eastern Daylight Time [-] File "D:\python projects\programs\zcfscheduler\loader.py", line 15, in ? 2005/03/24 17:14 AUS Eastern Daylight Time [-] run() <-- snip --> 2005/03/24 17:14 AUS Eastern Daylight Time [-] return getattr(reactor, 'listen'+self.method)(*self.args, **self.kwargs) 2005/03/24 17:14 AUS Eastern Daylight Time [-] File "C:\Python23\lib\site-packages\twisted\internet\default.py", line 283, in listenTCP 2005/03/24 17:14 AUS Eastern Daylight Time [-] p.startListening() 2005/03/24 17:14 AUS Eastern Daylight Time [-] File "C:\Python23\lib\site-packages\twisted\internet\tcp.py", line 594, in startListening 2005/03/24 17:14 AUS Eastern Daylight Time [-] raise CannotListenError, (self.interface, self.port, le) 2005/03/24 17:14 AUS Eastern Daylight Time [-] twisted.internet.error.CannotListenError: Couldn't listen on any:12005: (10048, 'Address already in use'). how can i get the server to stop, restart itself and release the port in the process? Regards, Grant McDonald Important notice: This message is intended for the individual(s) and entity(s) addressed. The information contained in this transmission and any attached, may be confidential and may also be the subject of legal privilege, public interest immunity or legal professional privilege. Any review, retransmission, dissemination or other use of, taking of any action in reliance upon this information by person or entities other than the recipient is prohibited and requires authorization from the sender. If you are not the addressee indicated in this message (or responsible for delivery of the message to such person) you may not copy or deliver this message to anyone. In such cases you should destroy this message and kindly notify the sender by reply email. WARNING: Although Infocomp has taken reasonable precautions so that no viruses are present in this e-mail, the company cannot accept responsibility for any loss or damage arising from the use of e-mail attachments.

1 0

[Twisted-Python] Twisted Names Deprecation Warning Fixes
by Peter Kropf March 23, 2005

March 23, 2005

In the Twisted pseudo-sprint yesterday, I made some changes to the doc/names/examples/testdns.py and twisted/names/dns.py to clean up the deprecation warnings. Attached are two svn diffs of the changes made. I've run the trials for twisted names and everything passes with the exception of testIPv6. That was skipped since I'm not using IPv6 on my laptop. Would someone please review them and commit the changes to SVN? There's also a deprecation warning in twisted/names/client.py that I'll try to get to while at the pycon... Thanks. - Peter peter@think:/space/src/twisted/Twisted> svn diff doc/names/examples/testdns.py Index: doc/names/examples/testdns.py =================================================================== --- doc/names/examples/testdns.py (revision 13290) +++ doc/names/examples/testdns.py (working copy) @@ -1,9 +1,8 @@ #!/usr/bin/env python import sys -from twisted.names import client +from twisted.names import client, dns from twisted.internet import reactor -from twisted.protocols import dns r = client.Resolver('/etc/resolv.conf') @@ -27,6 +26,10 @@ if __name__ == '__main__': import sys + if len( sys.argv ) < 2: + print 'usage: testdns.py domain_name' + sys.exit( 1 ) + r.lookupAddress(sys.argv[1]).addCallback(gotAddress).addErrback(gotError) r.lookupMailExchange(sys.argv[1]).addCallback(gotMails).addErrback(gotError) r.lookupNameservers(sys.argv[1]).addCallback(gotNameservers).addErrback(gotError) peter@think:/space/src/twisted/Twisted> svn diff twisted/names/dns.py Index: twisted/names/dns.py =================================================================== --- twisted/names/dns.py (revision 13294) +++ twisted/names/dns.py (working copy) @@ -1086,7 +1086,8 @@ m = Message() m.fromStr(data) if self.liveMessages.has_key(m.id): - d = self.liveMessages[m.id] + d, timeout_call = self.liveMessages[m.id] + timeout_call.cancel( ) del self.liveMessages[m.id] # XXX we shouldn't need this hack of catching exceptioon on callback() try: @@ -1123,8 +1124,13 @@ self.resends[id] = 1 m = Message(id, recDes=1) m.queries = queries - d = self.liveMessages[id] = defer.Deferred() - d.setTimeout(timeout, self._clearFailed, id) + + from twisted.internet import reactor + d = defer.Deferred() + timeoutCall = reactor.callLater( + timeout, + lambda: d.called or self._clearFailed( d, id ) ) + self.liveMessages[id] = ( d, timeoutCall ) self.writeMessage(m, address) return d

2 3

[Twisted-Python] Note for 2.0 package maintainers
by Eric Mangold March 23, 2005

March 23, 2005

If you're creating 2.0 packages there is one little thing you need to do to make Zsh tab-completion work automatically. The file twisted/python/_twisted_zsh_stub needs to be installed in the zsh site-functions directory. Of course, where the site-functions directory is located is system dependant. If you have zsh installed, you can see where it is with this command: zsh -f -c 'echo ${(M)fpath:#/*/site-functions}' -Eric

1 0

[Twisted-Python] Re: [Twisted-commits] r13295 - WSGI input/error streams, pausing when write buffer is full, and file wrapper.
by Andrew Bennetts March 23, 2005

March 23, 2005

On Tue, Mar 22, 2005 at 03:50:59PM -0700, James Knight wrote: [...] > + > +def callInReactor(__f, *__a, **__kw): > + from twisted.internet import reactor > + queue = Queue.Queue() > + reactor.callFromThread(__callFromThread, queue, __f, __a, __kw) > + result = queue.get() > + if isinstance(result, failure.Failure): > + result.raiseException() > + return result > + > +def __callFromThread(queue, f, a, kw): > + result = defer.maybeDeferred(f, *a, **kw) > + result.addBoth(queue.put) I recall posting example code to do a similar job to the mailing list recently, and also someone on the list (or irc channel?) asking how to do this (make the thread that does callFromThread block on the result). This should probably be turned into a proper API. -Andrew.

1 0

[Twisted-Python] HTTP File Posting and Retrieving
by Steve Milner March 21, 2005

March 21, 2005

Hello List, I'm looking to use Twisted to download and upload files via HTTP. I've looked at the examples for web.client but am still a bit confused (and am wondering if maybe there is a different class I should be working with). My questions are: 1.) How can I upload a file 2.) How can I download a file and get the headers for parsing (size and whatnot). Thanks in advance, Steve

1 1

[Twisted-Python] multi-threading in twisted
by Joachim Boomberschloss March 21, 2005

March 21, 2005

Hi, I just want to make sure I understand what the docs say: can reactor.callFromThread be called from any thread? and, is it blocking? In my application I have a thread that comes up with data to process every once in a while, and I want that processing to be done in the main loop because it may involve network communication, so I planned to copy the data and then call reactor.callFromThread with the method to do the processing. Does this sound sound? Thanks, Joe __________________________________ Do you Yahoo!? Yahoo! Mail - Easier than ever with enhanced search. Learn more. http://info.mail.yahoo.com/mail_250

1 0

[Twisted-Python] Creating a dynamic FTP server
by Jason Rust March 21, 2005

March 21, 2005

Hi, I've been poking around in the code and documentation for a bit now and wanted to see if the following project would be doable with twisted before I actually started coding it: We want a FTP server where all the files and directories are "virtual." The actual files would be in a database and the folders shown to a user would be based on the time of day and who the user is. When getting or putting a file it would pull or put it into the database backend. Would something like this be possible with the twisted FTP module? Thanks, -Jason

3 2

[Twisted-Python] Reminder: Australian Twisted Sprint, Hobart, 1-3 April
by Mary Gardiner March 21, 2005

March 21, 2005

Just a reminder for people that the Australian Twisted Sprint is in two weekend's time. Details are available from the wiki at http://users.puzzling.org/users/mary/TwistedSprint/moin.cgi/FrontPage We'd appreciate hearing from people who are coming, as per http://users.puzzling.org/users/mary/TwistedSprint/moin.cgi/SprintAttendees . Also, there's a few people on the "maybe" list who we have no way to contact to confirm their attendence -- an email would be great. Thanks, Mary

1 0

[Twisted-Python] clients of perspective brokers
by Joachim Boomberschloss March 20, 2005

March 20, 2005

I have a question about clients of perspective brokers: I want the processes of my application to be able to connect arbitrarily to other processes in a local network without assuming their existence or location in advance. In the Twisted examples and how-tos, however, the idiom seems to be defining a-priori the other clients and servers in the network (usually before calling reactor.run, or as part of an application�s configuration). My question is, are there any restrictions on calling reactor.connectTCP (or similar) arbitrarily during the execution of the program, is there any cleanup to do when not requiring the other party anymore, etc. Also, it is unclear to me how reconnection works for perspective broker clients. If a connection is dropped by the server, and then a clients tries to make a method call on a remote object, will the client factory try to reconnect before making the request, or will the request fail, and re-connection be attempted the next time, etc.? Thanks, Joe. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

3 3