Mailman 3 May 2009 - Twisted

Re: [Twisted-Python] Another approach to allowing __init__ to work with Deferreds
by Terry Jones May 12, 2009

May 12, 2009

Hi Drew. Thanks for the reply. >>>>> "Drew" == Drew Smathers <drew.smathers(a)gmail.com> writes: Drew> Somewhere, someplace something has to get a reference to the object Drew> and it seems to me you're trying to prevent that something from Drew> calling methods on the instance of the object before it's ready. So Drew> why not just defer providing the reference instead of wrapping Drew> methods and intercepting calls? To illustrate, here's a simple Drew&… [View More]

1 0

[Twisted-Python] Announcing Twotp 0.6
by Thomas Hervé May 11, 2009

May 11, 2009

Hello all! I'm pleased to announce version 0.6 of Twotp [1]. What is Twotp? It's an implementation of the Erlang protocol using Python and Twisted, similar with the built-in Jinterface [2] or py_interface [3]. After some hibernating time, I've focused on implementing links and monitoring of Erlang processes. Now it's even more complete than Jinterface, which doesn't implement monitoring. After implementing the lower layer, it made me realize that the current API was all but convenient. So I … [View More]

2 1

[Twisted-Python] Twisted plugins and py2exe
by Gabriel Rossetti May 10, 2009

May 10, 2009

Hello everyone! I used py2exe on a Twisted app of mine and I get this exception : Traceback (most recent call last): File "proj\service.pyc", line 1308, in connectionMade File "proj\service.pyc", line 1640, in _register File "proj\service.pyc", line 504, in loadPlugins File "twisted\plugin.pyc", line 200, in getPlugins --- <exception caught here> --- File "twisted\plugin.pyc", line 179, in getCache exceptions.AttributeError: ZipPath instance has no attribute 'setContent' … [View More]

3 6

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com May 10, 2009

May 10, 2009

1 0

[Twisted-Python] Twisted HTTP 1.1 Web Client
by Reza Lotun May 8, 2009

May 8, 2009

Hi, Just curious, but what's the roadmap for the new web client (http://twistedmatrix.com/trac/ticket/886)? As I understand it only the finalization of a public api remains? Cheers, Reza -- Reza Lotun +44 (0)7521 310 763 rlotun(a)gmail.com

2 1

[Twisted-Python] how to authenticate SMPT server in Twisted Framework
by ganesh gunasekaran May 8, 2009

May 8, 2009

Hi gp, I'm very new to Twisted Framework , I wrote a very simple code to send mails concurrently, using SMTP mail server the code as below from twisted.mail.smtp import sendmail from twisted.internet import reactor from twisted.python.log import err import time MAILSERVER = 'mail.xxx.com' listTo = ['ganesh.xxx(a)gmail.com', 'xxjango.py(a)gmail.com', ' lovely_xxx(a)yahoo.co.in'] FROM = 'gxxxkaran(a)xxxx.com' MSGBODY = "hi this is test mail" a= time.time() done = sendmail(MAILSERVER, FROM, … [View More]

2 1

Re: [Twisted-Python] Question about deferreds
by Doug Farrell May 7, 2009

May 7, 2009

Glyph, Thank you very much for your response, it has provided me with material that will help me take a big step in my Twisted education. > > Deferreds are not about breaking your work up into steps to save time. > They're about breaking it up so that different systems can deal with > the > same asynchronous event in their own way. > > For example, let's say you're working on a REST client that deals with > invoices. The server gives you data at a … [View More]

2 1

[Twisted-Python] how do i do that
by sophismus＠gmx.de May 7, 2009

May 7, 2009

hey, i'm quite new in python and twisted and did things wrong from the start now i have to reconsider my prog. first i thought of an easier example with almost the same functionality. functionality: client connects, sends info, all infos from all clients are concurrently processed, a special client with the right info at the right time gets a special msg i got this: it is thought that a client writes an int as birthday back... from twisted.internet import protocol, reactor, defer, threads … [View More]days = [] allprotocols = [] class BServerProtocol(protocol.Protocol): birthday = 0 def connectionMade(self): print "puh" global allprotocols allprotocols.append(self) print "allprotocols",allprotocols def congratulate(self): self.transport.write("congratulations") return 'done' def dataReceived(self,data): print data if int (data)>0: self.birthday = int(data) print "self.bday:",self.birthday global days days.append(int(data)) self.transport.write("pff") # self.factory.deferred.callback("uiuiui+1") if data == "thanks": self.write("well done") class BServerFactory(protocol.ServerFactory): protocol = BServerProtocol def __init__(self): self.deferred = defer.Deferred() def proveday(): day = 5 #of may, think of this date as new decided #from a white rabbit every time it runs ret = -1 found = False i = 0 length = len(allprotocols) while (i<length and found == False): birthday = allprotocols[i].birthday print "birthday:",birthday if birthday == day: print 'bday==day' print "allprotocols[i]:",allprotocols[i] found =True ret = i allprotocols[i].congratulate() i+=1 return ret def startprove(): r = proveday() bsf = BServerFactory() #bsf.deferred.addCallback(registerSuccess) reactor.listenTCP(55555,bsf) reactor.callInThread(startprove) #commands =[(startprove,[],{})] #commands.append(startcong)#,[],{}) #threads.callMultipleInThread(commands) reactor.run() _______________________ allprotocols[i].congratulate() should get in connection with the client again, but nothing happens, when another client connects a lot of congratulations are thrown, it seems transport buffers. i tried a lot of things, with no better result, maybe i can get some hints to get a solution... thx stefan -- Psssst! Schon vom neuen GMX MultiMessenger gehört? Der kann`s mit allen: http://www.gmx.net/de/go/multimessenger01 [View Less]

6 6

[Twisted-Python] How can I change max simultaneously connection in twisted
by 孙绍轩 May 6, 2009

May 6, 2009

Hello all I just wrote a small code to holding many http requests until server's status change. Everything is fine but when simultaneously connection up to 1k+, the server code didn't response to new connection. I've set ulimit -n's value to 65535 and the code using epollreactor mode. Are there have some limitations? How can I configure to support more connections? BTW: I use Python 2.4.3 on CentOS 5 64-bit. follow is the code, Thanks for you help! from twisted.internet import epollreactor … [View More]epollreactor.install() from twisted.internet import reactor, defer, threads from twisted.web import server, resource, client from twisted.python import log from time import sleep import sys def checkstatus_wait(laststatus): timeout = 25 while timeout > 0: if int(laststatus) < status: return True else: timeout -= 1 sleep(1) return False class Simple(resource.Resource): isLeaf = True def render_GET(self, request): if "cmd" in request.args: cmd = request.args['cmd'][0] else: cmd = "" if cmp(cmd,"listen") == 0: if "laststatus" in request.args: last = request.args['laststatus'][0] if int(last) < status: return "ret=1\nerrstring=need update\nlaststatus=%d\n\n" % status else: d = threads.deferToThread(checkstatus_wait, last) d.addCallback(self.testReturn, request) return server.NOT_DONE_YET else: return "ret=1\nerrstring=need update\nlaststatus=%d\n\n" % status elif cmp(cmd,"updatestatus") == 0: if status < 1000000: status += 1 else: status = 0 #client.getPage('http://211.157.0.47/videochat/listen?cmd=updatestatus') return "ret=1\nerrstring=success update\n\n" else: return "ret=-1\nerrstring=wrong cmd\n\n" def testReturn(self, ret, request): if ret: request.write("ret=1\nerrstring=need update\nlaststatus=%d\n\n" % status) else: request.write("ret=0\nerrstring=no need update\n\n") request.finish() status = 0 log.startLogging(sys.stdout) site = server.Site(Simple()) reactor.suggestThreadPoolSize(5000) reactor.listenTCP(8080, site) reactor.run() -- 孙绍轩 Yorgo Sun China Distance Education Holdings Limited 18th Floor, XueYuan International Tower, No.1 ZhiChun Road, HaiDian District, Beijing, China 100083 Phone: +8610-8231-9999 Ext 708 *********************************************************** CONFIDENTIALITY CAUTION: This message is intended only for the use of the individual or entity to whom it is addressed and contains information that is privileged and confidential. If you, the reader of this message, are not the intended recipient, you should not disseminate, distribute or copy this communication. If you have received this communication in error, please notify us immediately by return email and delete the original message. [View Less]

3 4

[Twisted-Python] Question about deferreds
by Doug Farrell May 5, 2009

May 5, 2009

Hi all, I consider myself a Twisted "newbie", so this might seem like a simple couple of questions, hope you'll indulge me. When I first began experimenting with deferreds I was breaking up tasks into sequences of steps something like this: d = Deferred() d.succeed(True) d.addCallback(step1) d.addCallback(step2) d.addCallback(step3) etc. When this runs I was under the mistaken impression that at the end of each callback (step1, step2, step3, etc) the deferred would yield … [View More]

2 1

Re: [Twisted-Python] RTFM about custom reactors
by andrewfr_ice＠yahoo.com May 5, 2009

May 5, 2009

Message: 3 Date: Mon, 4 May 2009 02:15:25 -0400 From: Mohamed Lrhazi <lrhazi(a)gmail.com> Subject: [Twisted-Python] RTFM about custom reactors To: twisted-python(a)twistedmatrix.com Message-ID: <c4ab18df0905032315m4f0b3f85vf900adddd7557000(a)mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1 Hi Mohamed: >I was wondering if someone could point me in the right direction.... >I am attempting to connect a proprietary network OS plugin system to >… [View More]

1 0

[Twisted-Python] addCallback0 convenience method
by Terry Jones May 5, 2009

May 5, 2009

There are two minor niggles I frequently run into when writing code with deferreds and callbacks: 1. I very often write callbacks that ignore the incoming result: I tend to write this as def cb(_): ... 2. I sometimes write functions that I want to call both from within a callback chain and when not in a callback chain. Usually this is another case of writing def x(_, arg1, arg2) etc. and documenting that the extra unused arg is for when the function is called as part of a … [View More]

3 2

[Twisted-Python] RTFM about custom reactors
by Mohamed Lrhazi May 4, 2009

May 4, 2009

Hello all, I was wondering if someone could point me in the right direction.... I am attempting to connect a proprietary network OS plugin system to Python, to allow for writing traffic handling plugins in Python, and was wondering what it would take to connect such a system into Twisted, thus allowing the writing of normal twisted apps on top.... Is it matter of learning and writing a custom reactor? My current understanding is that the reactor is what detects and dispatches events... but … [View More]

2 2

[Twisted-Python] sending out RAW IP packets using sockets
by Lars Reimann May 4, 2009

May 4, 2009

hello, this question may be a bit off-topic, but nevertheless I am stuck atm. I'm trying to send out RAW IP packets (incl. full header) which i obtained previously using packet capture. It has to be RAW IP packets because I want the kernel to do ARP lookup stuff, etc for me . So an obtained packet should go through the normal lookup process. I've tried to use RAW sockets in python for this, but it constantly tells me, that the "Protocol (is) not supported" using this code: ------- self.… [View More]

1 0

[Twisted-Python] design question
by Pet May 4, 2009

May 4, 2009

Hi, I've written my first Twisted Server, which accept request from clients, does DB queries, sends response back and logs some events in DB. I wonder, if my design was good. Basically, I have a Factory where is db connectionPool created and Protocol(basic.LineReceiver), which processes requests and sends data back. In Protocol, I instantiate my utility object which I try to keep independent from twisted framework, so it can be easily used somewhere else or can be replaced by other class. … [View More]

1 0

Re: [Twisted-Python] cx_Oracle, Twisted - rollback failed
by Steve Lewis May 3, 2009

May 3, 2009

> Is there some way it can determine that the connection has been lost aside > from trying the rollback (and is compatible with other DB-API 2.0 modules)? > > Jean-Paul Oops, I actually did see my message going through now. Sorry for the duplicate. To answer your question, I was under the impression that the cp_reconnect would check to see if the connection existed already. Is there a way to have it check that query before rolling back? I'm guessing other databases/apis don't … [View More]

2 1

[Twisted-Python] May 2nd Sprint
by Jean-Paul Calderone May 3, 2009

May 3, 2009

Hello all, The sprint yesterday went quite well. He's a quick peek at what everyone was up to: * Drake Smith and Itamar Shtull-Trauring spent time fixing problems with our documentation and addressing a potential problem with the __str__ of DelayedCall * Dave Sturgis worked on adding frame-drop notification to LoopingCall * Thomas Hervé contributed remotely with code for the Conch "session" branch and tests for a regression fix in Conch's ssh agent support * Michael … [View More]

1 0

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com May 3, 2009

May 3, 2009

1 0

Re: [Twisted-Python] TCP Proxy scalability issue
by Mohapatra Saurav May 2, 2009

May 2, 2009

Glyph, Thanks for the response. We're trying to proxy multiple VNC processes for a screen sharing application. So though CPU load is low, the data being pushed out is pretty high. We have around 16Kb of data being pushed out per second per client for a normal case. The test server is running on EC2, but we see the degradation in the production deployment (on physical machines running on a cogent fiber connection). I was thinking it might have something to do with amount of data being written … [View More]and number of simultenous writes issued. Something to do with transport.write() I'll take a look at txloadbalancer. I can make a benchmarking application available to you, please mail me off the list at saurav.mohapatra(a)dimdim.com Regards Saurav -----Original Message----- From: twisted-python-bounces(a)twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] On Behalf Of twisted-python-request(a)twistedmatrix.com Sent: Friday, May 01, 2009 12:00 PM To: twisted-python(a)twistedmatrix.com Subject: Twisted-Python Digest, Vol 62, Issue 2 Send Twisted-Python mailing list submissions to twisted-python(a)twistedmatrix.com To subscribe or unsubscribe via the World Wide Web, visit http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python or, via email, send a message with subject or body 'help' to twisted-python-request(a)twistedmatrix.com You can reach the person managing the list at twisted-python-owner(a)twistedmatrix.com When replying, please edit your Subject line so it is more specific than "Re: Contents of Twisted-Python digest..." Today's Topics: 1. Re: TCP Proxy scalability issue (glyph(a)divmod.com) ---------------------------------------------------------------------- Message: 1 Date: Fri, 01 May 2009 15:54:56 -0000 From: glyph(a)divmod.com Subject: Re: [Twisted-Python] TCP Proxy scalability issue To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Message-ID: <20090501155456.12555.1002021953.divmod.xquotient.10153(a)weber.divmod.com > Content-Type: text/plain; charset="utf-8"; format="flowed" On 02:02 pm, saurav.mohapatra(a)dimdim.com wrote: >Worker processes W1...n run listening on P1..n on the loopback and one >router process (twisted based) runs on public port P0 exposed to the >real world. I haven't used it myself, but that sounds a bit like txloadbalancer: https://launchpad.net/txloadbalancer Are you using that? >The clients connect to P0 and the first few bytes they send indicates >which worker process they wish to connect to. The Twisted protocol >implementation then creates a relay TCP connection to loopback worker >process port and after that forwards received data on P0 to the worker >port on loop back and sends back data received from worker port to the >external connection. >We're noticing significant degradation / starvation of the clients >under >load (around 25 concurrent connections are enough to simulate this). >We're running on Linux (CentOS 5.2) using python 2.5 and Twisted latest >source tarball using the epoll reactor and all settings are default. >Each connection sends back around 8-24kb data per second. While every application is a unique beast in terms of performance tuning, these numbers all sound surprisingly low to me for something as simple as a TCP proxy. Are you saturating your CPU? What is the load like on the box in question, both from the Twisted proxy and from the other processes? Can you provide a benchmark that we can run somewhere else, to demonstrate the issue you're having? >My question is are there any "production" settings I need to do >(threadpool etc.) to get the maximum out of twisted. Nothing that you described will make any use of threads - is there some reason you mention threadpool size? If you *are* using threads then perhaps you shouldn't be, and they're causing performance problems :) However, aside from reactor selection, Twisted is designed to have very few knobs to turn; it has one button for performance tuning and we push it before it leaves the factory. So if there's a performance problem, the issue is that we need to optimize something in Twisted, or you need to optimize something in your application. You may be able to tweak various linux kernel parameters to be able to improve things a bit, but if you're running into problems at 25 connections, it doesn't sound like you're running into kernel issues to me. ------------------------------ _______________________________________________ Twisted-Python mailing list Twisted-Python(a)twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python End of Twisted-Python Digest, Vol 62, Issue 2 ********************************************* [View Less]

2 1

[Twisted-Python] TCP Proxy scalability issue
by Mohapatra Saurav May 1, 2009

May 1, 2009

Hi, We're currently running a TCP proxy / port multiplexer written in Twisted to route to multiple worker server processes running on loopback ports. The architecture is something like this Worker processes W1...n run listening on P1..n on the loopback and one router process (twisted based) runs on public port P0 exposed to the real world. The clients connect to P0 and the first few bytes they send indicates which worker process they wish to connect to. The Twisted protocol … [View More]

2 1