Mailman 3 March 2005 - Twisted

Re: [Twisted-Python] How to accept connections faster
by Andrzej Leszczynski 16 Mar '05

16 Mar '05

Hi, > print "Looping..." > for i in range(10000000): > pass > print "Looping done." regardles I am not sure the reason of the loop and how it fits into the design of Reactor, I wonder why it is *only* 2 seconds. On my laptop 1.1GHz, 256MB RAM it takes a way more to complete the loop and unblock the Reactor's "select". Second comment is why not xrange(10000000). With range my python2.3 starts allocating xxMB of memory. What machine,OS,python version you use? A.

1 0

[Twisted-Python] declare pb.Copyable only once
by Henning.Ramm＠mediapro-gmbh.de 16 Mar '05

16 Mar '05

Hi there! At the moment I'm trying to wrap my mind around pb objects. I learned from the pb.Copyable howto (http://twistedmatrix.com/documents/current/howto/pb-copyable) that I must setUnjellyableForClass every object that could be sent from a client. I need a simple empty utility class like this: class Info(pb.Copyable): def __init__(self, obj=None): if obj: self.__dict__.update(obj.__dict__) Because I can use such in server as well as in client, I defined the class in the server and imported it in the client. Nevertheless I get an InsecureJelly exception in the server. If I setUnjellyable... it in the server as in pb.setUnjellyableForClass(Info, Info) I get an AttributeError: Info instance has no attribute 'unjellyFor' Is there any way to declare this class only once and not separate for server and client? Best regards, Henning Hraban Ramm Südkurier Medienhaus / MediaPro Support/Admin/Development Dept.

1 0

[Twisted-Python] How to accept connections faster
by Sinang, Danny 16 Mar '05

16 Mar '05

Hello. I ran the (Twisted) server code below and it accepts a client socket connection every 1.3 seconds. The equivalent Synchronous socket code accepts one every 2 seconds. Can anyone here suggest any code improvements for my Twisted server to accept connections faster ? I've include the Synchronous socket server code and the client generator code below as well. # ======================================================================== ================== # Twisted server code # ======================================================================== ================== from twisted.protocols import basic from twisted.internet import reactor, protocol import time class MyChat(basic.LineReceiver): def connectionMade(self): print "Got a connection from on " + time.asctime() self.transport.write("Welcome. You are client number : ") print "Looping..." for i in range(10000000): pass print "Looping done." self.transport.loseConnection() def connectionLost(self, reason): pass def lineReceived(self, line): for c in self.factory.clients: c.message(line) def message(self, message): self.transport.write(message + '\n') def __init__ (self) : self.x = 1 self.duration = 60 print "Starting Twisted Async Server" print factory = protocol.ServerFactory() factory.protocol = MyChat factory.clients = [] reactor.listenTCP(7878,factory,10) reactor.run() # ======================================================================== ========= # Synchronous socket server code # ======================================================================== ========= from socket import * import sys import time s = socket(AF_INET, SOCK_STREAM) s.bind(('', 7878)) s.listen(10) print 'Login Server started on port 7878' x = 1 while 1: client,addr = s.accept() print str(x) + ". Got a conection from " + str(addr) + " on " + time.asctime() print "Looping..." for i in range(10000000): pass print "Loop done. Ready for connection..." print client.sendall ("Welcome. You are client number : " + str(x)) x += 1 # ======================================================================== ========= # Socket client code # ======================================================================== ========= from socket import * import sys import time s = socket(AF_INET, SOCK_STREAM) print 'Creating 20 clients...' x = 1 duration = 60 for i in range(20): print "Client " + str(x) + "." + " connecting to localhost at " + time.asctime() s = socket(AF_INET, SOCK_STREAM) s.connect (("127.0.0.1",7878)) print "Client Connected..." , time.asctime() s.sendall("Hello\r\n") data = s.recv(1024) print data x += 1 s.close() Regards, Danny

5 4

[Twisted-Python] Re: [Twisted-commits] r13185 - Applied patch from Ralph Meijer for issue 448
by James Y Knight 16 Mar '05

16 Mar '05

I'll pick on your commit message dizzyd, but this is a general request: Can people not post log messages like this? It makes it impossible to write a NEWS file if commits look like this. I cannot go through and look at every bug report to see what the commit actually did, and I also cannot go back and read all the svn diffs. That would just take way too long. Any commits that looked like this I just completely ignored, which is a shame. The messages don't need to be excruciatingly detailed, but please include at least *something* about what the change did, even if it's completely obvious from the patch, because I'm not looking at the patches: just the message. E.g., in this case "Applied patch from Ralph Meijer for issue 448: fixes digest authentication in Jabber client" would be great. James On Mar 14, 2005, at 3:19 PM, Dave Smith wrote: > Author: dizzyd > Date: Mon Mar 14 13:19:36 2005 > New Revision: 13185 > > Modified: > trunk/twisted/words/protocols/jabber/client.py > Log: > Applied patch from Ralph Meijer for issue 448

2 1

Re: [Twisted-Python] Twisted based LDAP protocol implemenation
by Andrzej Leszczynski 16 Mar '05

16 Mar '05

--- Jeff Pitman <symbiont(a)berlios.de> wrote: > On Tuesday 15 March 2005 17:10, Tim Terleg�rd wrote: > > > I am looking for a Twisted based LDAP protocol > > > implementation (both versions 2 and 3). Is > anybody > > > aware of such, even in very early development > phase. > > > > There is ldaptor. Not sure where to download it > though, but it's > > definately available in debian and friends. > > Home: http://tv.debian.net/software/ldaptor > Download: > http://ftp.debian.org/debian/pool/main/l/ldaptor/ldaptor_%{version}.tar.gz > > Something like that. Thanks for the link. Got it and will look at it. Just out of the curiosity - what would be a reason it is not integrated with Twisted core protocols? AndyL

3 2

[Twisted-Python] New feature details for NEWS
by James Y Knight 16 Mar '05

16 Mar '05

So, if anyone wants to expand on items in this section of the NEWS file, that'd be great. I think we could do with more of an explanation about our major new features. If there are other important changes that you feel belong in this list, don't be shy, add them. > Major new features > ----------------------- > - Replaced home-grown components system with zope.interface. > - Split Twisted into multiple pieces. > - Relicensed: Now under the MIT license, rather than LGPL. > - Python 2.4 compatibility fixes > - Major efficiency improvements in TCP buffering algorithm. > - Major efficiency improvements in reactor.callLater/DelayedCall. > - Half-close support for TCP/SSL. (loseWriteConnection). Also, I somewhat punted on the trial section of the NEWS file. I'm really not sure what the new features are since 1.3. I know everything changed 500 times, but I don't know what the actual feature changes were. It was hard to tell from the commit messages, and I don't remember what trial was like back in 1.3 days. If someone more familiar with it would like it to say anything other than "Rewritten, a bunch of bugs fixed, a few more added", have at it. James

1 0

[Twisted-Python] how to stop dataReceived while waiting a deferred to fire?
by Andrea Arcangeli 15 Mar '05

15 Mar '05

My code is like this: def stringReceived(string): d = somesqlquery(string).addCallback(somecallback) Now before I can accept more data from the next stringReceived, I have to validate the "string" with the "somecallback". I don't want to read more data from the kernel _socket_ until I've the result of somecallback. The example in the docs is like this: ------------------------- class FingerProtocol(basic.LineReceiver): def lineReceived(self, user): self.factory.getUser(user ).addErrback(lambda _: "Internal error in server" ).addCallback(lambda m: (self.transport.write(m+"\r\n"), self.transport.loseConnection())) [..] we would have to fetch result from a remote Oracle? Or from the web? ------------------------- But as far as I can tell the above also is broken and it won't wait for the deferred to fire, so two of those queries could be invoked if the client side is malicious and it sends two lines in one packet and that could screwup the server. It'll work only as long as the clients aren't malicious. I really can't see how the reactor could ever wait for the deferred returned by GetUser and how could lineReceived being throttled in the above example. So I think the above example needs fixing too. I need one of those below two solutions: 1) something like protocol.transport.removeReader() to call before returning from stringReceived (this can be allowed to be different for stringReceived and dataReceived, since stringReceived will have to stop the already read data too, and in turn for stringReceived it could be a protocol.removeReader() that will call protocol.transport.removeReader internally) 2) I could return a deferred from stringReceived and have the reactor deregister the fd corresponding to the transport that returned a deferred until the deferred fires, this is a less finegrined solution but it's quite handy to code with and it would match the Nevow style of waiting deferreds I could try to workaround in my app by stacking some local buffering with removeReader in the reactor, but it's probably simpler to fix twisted for it instead. There must be a way to throttle the input at the kernel level, the recv syscall must not be called while the deferred is outstanding. Please help thanks, I'm stuck on this right now...

6 19

[Twisted-Python] potential bug in the reactor's handling events loop
by Andrzej Leszczynski 15 Mar '05

15 Mar '05

Python 2.3, one of the latest Twisted version: I noted that under Linux there is not way to Control-C the reactor loop. After digging a little I found that following change helps: [root@d1501-0a10109c internet]# diff base.py{,.ori} > 302d301 < print "1",sysEvtTriggers 305d303 < print "2",`self._eventTriggers` 307,308c305 < for callable, args, kw in sysEvtTriggers[1]: < print "3" --- > > for callable, args, kw in sysEvtTriggers[0]: 317d313 < print "4" 320d315 < print "5",`eventType` Of course it is quick work around, not a permanent fix, but it really helps. Please advice, Andy

4 5

[Twisted-Python] How to uninstall Twisted 2.0?
by Yves Goergen 15 Mar '05

15 Mar '05

Hi everybody, I just installed Twisted 2.0.0a2 only to find it incompatible with the application I wanted. Now I need to go back to version 1.3 or whatever is the latest stable release before 2. Can I just install 1.3 over 2.0? How can I uninstall version 2, there's no prepared mechanism for this described where I can see it. TIA -- Yves Goergen "LonelyPixel" <nospam.list(a)unclassified.de> Unclassified NewsBoard Forum --> newsboard.unclassified.de Free (GPL), easy to use and install, secure, innovative! (PHP+MySQL)

8 16

Re: [Twisted-Python] Re: Twisted and the Posh Module
by Ed Suominen 15 Mar '05

15 Mar '05

Ah, so *that* is why the SANE GUI frontend I'm writing [1] pauses scanning and window events when it's using PIL to generate a thumbnail of the last page scanned, despite my use of deferToThread. A unit test I wrote with time.sleep() passed just fine, and I threw my hands up and walked away. Thanks for shedding light on what was very confusing and frustrating behavior. [1] spdf: "Scan to PDF," GPL. A Twisted-driven application with, shockingly, absolutely no networking. Coming soon to a download URL near you... --- Ed Suominen Registered Patent Agent Open-Source Software Author (yes, both...) Web Site: http://www.eepatents.com On Monday 14 March 2005 02:11 pm, twisted-python-request(a)twistedmatrix.com wrote: > My goal is somewhat similar in that I have multiple CPU's that aren't > being used very much, but the other problem is that certain libraries > I use (PIL) grab the GIL for long-running operations like image > modifications. I can thread them off, but it still liberally acquires > the GIL and makes the server unresponsive for the duration of the > operation.

4 3