Mailman 3 April 2003 - Twisted

[Twisted-Python] Twisted 1.0.4 - Figures in the manual
by Francois Pinard April 23, 2003

April 23, 2003

Hello, Twisted people. In the manual found as `doc/howto/book.ps', some figures are inserted in the text with a 90 degree rotation. Despite this is not a great evil, they would appear better if not rotated. There is one on page 10 (1.1 High-Level Overview of Twisted); another on page 76 (4.6.2 Deferreds - Visual Explanation); another on page 147 (6.3.3 The Diagram), and maybe others, I did not try to be exhaustive here. The figure on page 10 drops off the page, however, and this is more … [View More]

2 1

[Twisted-Python] flow.Zip
by Philippe Lafoucrière April 23, 2003

April 23, 2003

Clark, just a mail to remind you to code flow.zip. with merge :[1,Cooperate(),2] + [3,4] = [1,3,4,2] with zip, it should be [1,Cooperate(),2] + [3,4] = [(1,3)(2,4)] Hmm, By the way, Cooperate() isn't clear, even with the doc... :-( could you explain me that please ? thx Philippe

2 5

[Twisted-Python] UDP changes
by Itamar Shtull-Trauring April 23, 2003

April 23, 2003

I checked in some changes to UDP in CVS that may very will improve performance significantly. Or not. Could someone check and see if there's any difference? Code for a UDP throughput benchmark would also be much appreciated. -- Itamar Shtull-Trauring http://itamarst.org/ http://www.zoteca.com -- Python & Twisted consulting

3 2

[Twisted-Python] Broken HOWTO documentation
by Michael McLay April 23, 2003

April 23, 2003

I have been trying to get the examples in section 2.4 of the HOWTO to work. It looks like the documentation is out of date with the current state of the software in the CVS. starting with 2.4.2, the "app.theApplication" returns the error >>> a = app.theApplication Traceback (most recent call last): File "<stdin>", line 1, in ? AttributeError: 'module' object has no attribute 'theApplication' >>> dir(app) ['Accessor', 'Application', 'ApplicationService', '… [View More]

2 1

[Twisted-Python] t.i.process uid/gid suckiness
by Tommi Virtanen April 23, 2003

April 23, 2003

The handling of uid/gid changes and SIGCHLD all around twisted are in an appalling state. I've tried to bitch at the right people, but that hasn't helped, so I'm taking the more heavyhanded approach of just fixing stuff. Let the code talk. However, I still don't consider myself an authority on what Twisted should do, don't have that much time longer term (I will not maintain this code permanently), and don't know what silly limitations the windows port of python has. So, I'll try to … [View More]write down what changes I'm planning on doing, so you have a chance of vetoing. If I get no comments, I'll just basically go ahead and commit what I happen to want to, touching a many areas of mktap/t.i.process/procmon/whatever. If you want to avoid a de facto hijack-fix-abandon process, TALK NOW 0) make the t.i.process setuid/setgid code actually work (see patch; BTW is the initgroups part really needed? I feel my pure-python 6-liner does the same thing.) 1) make all places that take uid/gid just silently take in strings, too, and use pwd/grp to convert them to uids and gids as necessary 2) make .taps store uids and gids as strings 3) defer procmon startup so the processes are forked only after setuid has happened. 4) try to fix the child process races, atleast including deferring signal-triggered processing to happen outside the actual signal handler, rewriting the reap logic silliness, and stopping procmon from trying to kill reaped children. 5) fix whatever problems I noticed while fixing the above 6) whatever else I feel like improving at the time 7) ??? 8) profit! diff -u -u -r1.50 process.py --- twisted/internet/process.py 10 Mar 2003 20:16:57 -0000 1.50 +++ twisted/internet/process.py 11 Mar 2003 13:08:00 -0000 @@ -33,10 +33,25 @@ pty = None try: - from initgroups import initgroups import pwd + try: + from initgroups import initgroups + except: + import grp + def initgroups(username, dummy): + l=[] + for groupname, password, gid, userlist in grp.getgrall(): + if username in userlist: + l.append(gid) + os.setgroups(l) + def switch_uid(uid, gid): + os.setgid(gid) + initgroups(pwd.getpwuid(uid)[0], gid) + os.setuid(uid) except: - def initgroups(*args): pass + def switch_uid(uid, gid): + os.setgid(gid) + os.setuid(uid) from twisted.persisted import styles from twisted.python import log, failure @@ -229,9 +244,7 @@ os.chdir(path) # set the UID before I actually exec the process if settingUID: - os.setgid(gid) - initgroups(pwd.getpwuid(uid)[0], gid) - os.setuid(uid) + switch_uid(uid, gid) os.execvpe(command, args, environment) except: # If there are errors, bail and try to write something @@ -482,9 +495,7 @@ # set the UID before I actually exec the process if settingUID: - os.setgid(gid) - initgroups(pwd.getpwuid(uid)[0], gid) - os.setuid(uid) + switch_uid(uid, gid) os.execvpe(command, args, environment) except: stderr = os.fdopen(1, 'w') -- :(){ :|:&};: [View Less]

5 7

[Twisted-Python] Any papers or slides etc available from PyCon?
by Thomas Weholt April 23, 2003

April 23, 2003

Hi, I was hoping some of the things at PyCon related to Twisted would be available for all of us who didn't have a chance to attend. Especially the part about deployment of Twisted websites. Any information about this ?? Thomas

3 2

[Twisted-Python] Twisted 1.0.4 - `html' .vs. `xhtml'
by pinard＠iro.umontreal.ca April 23, 2003

April 23, 2003

Hello. In the `doc/' hierarchy of the Twisted 1.0.4 distribution, some `.html' files have links to `.xhtml' files, where apparently `.html' files by the same base exist. Should `.html' rather point to `.html' files whenever possible? Here are possible points to visit: * `doc/index.html' has references to `howto/index.xhtml', 'man/index.xhtml', `specifications/index.xhtml' and `vision/index.xhtml'. * `doc/howto/woven.html' refers to `listings/HelloWorld/HelloWorld.html' correctly, yet … [View More]

7 14

[Twisted-Python] serial port blocking on Linux
by Eric C. Newton April 22, 2003

April 22, 2003

Twisted 1.0.4 Linux Redhat 9.0 I have evidence of the SerialPort class blocking. strace says write operations are blocking and always return the number of bytes to be written. The problem goes away if I set the file descriptor on the object to non-blocking using posixy controls. Test program below. -Eric #! /usr/bin/python from twisted.internet import reactor from twisted.internet.serialport import SerialPort from twisted.protocols.basic import LineReceiver import fcntl, os import time … [View More]

1 0

[Twisted-Python] perspective broker question
by Egor Cheshkov April 22, 2003

April 22, 2003

Hello everyone! Newbie question. I need to make a blocking call in perspective_X() method. So I put my call into separate thread and get a deferred wich will be called back when data available. But deferred could not be returned from perspective_X method (not sure). My question is how can I implement perspective_X method wich getting data from deferred? -- Egor Cheshkov <egor(a)iplus.ru>

2 1

[Twisted-Python] UDP servers and socket queues
by Phil Mayers April 22, 2003

April 22, 2003

All, I'm porting an app over which had it's own async job thingy, similar to Twisted's most basics - I now want it to run as a long-running server with multiprotocol support, and Twisted is the logical choice. The core of the app is a UDP-based RPC client (bespoke stuff) which does a periodic flurry of activity (plus on-demand stuff - that's the plan for later anyway). The problem is that the flurry of activity fills the UDP socket queue, and I think some simple changes to SelectReactor (… [View More]which I've done locally via a nasty/messy subclass) which could be useful to others will fix it: from twisted.internet import reactor from twisted.internet import client class RPCProtocol(protocol.DatagramProtocol): def __init__(self); # setup some icky stuff for retries etc here def datagramReceived(self, data, addr): pdu = self._decode(data) dfurred = self.outstanding[pdu.uuid] del self.outstanding[pdu.uuid] dfurred.callback(pdu) def sendPDU(pdu, addr): self.outstanding[pdu.uuid] = defer.Deferred() self.writeMessage(pdu.encode(), addr) return self.outstanding[pdu.uuid] class endpoint: def __init__(self, protcol, host): self.protocol = protocol self.addr = resolve(host) self.actions = load_from_config(host) def go(self): pdu = self.create_pdu(self.actions.pop(0)) self.protocol.sendPDU(pdu, self.addr).addCallback(self.handle) def handle(self, pdu): func = self.actions.pop(0) func(pdu) pdu = self.create_pdu(self.actions.pop(0)) self.protocol.sendPDU(pdu, self.addr).addCallback(self.handle) Anyway, you get the idea. The code fires up, and you get: proto = RPCProtocol() for host in hosts: ep = endpoint(proto, host) ep.go() ...the problem is that the responses come back too fast - they overflow the UDP socket buffer after a couple of hundred packets. So, try this: proto = RPCProtocol() for host in hosts: ep = endpoint(proto, host) reactor.callLater(0, ep.go) ...except that the default SelectReactor will always execute any pendingTimedCalls before calling doIteration (where the select() call actually happens). Ok, so inherit from SelectReactor and override mainLoop - except that doIteration will only execute one doRead for each ready socket (there's one socket - there are potentially thousands of hosts and I don't want to overload either select()'s performance characteristics with the number of fd's or the fd limit). My old code did this: while running: while 1: rs, ws, xs = select.select([self.fd,], [], [], self.timeout()) if self.fd in rs: self.doRead() else: break for p in self.pending: p() ...i.e. empty the socket queue before doing *any* callback activity. I can do this with a suitable reactor - it's just messy. Is it worthwhile a flag in the core SelectReactor to change the preferences for network/callback behaviour and the rate of network activity? Just a thought. Twisted is excellent work - keep it up! -- Regards, Phil +------------------------------------------+ | Phil Mayers | | Network & Infrastructure Group | | Information & Communication Technologies | | Imperial College | +------------------------------------------+ [View Less]

4 6