Mailman 3 June 2007 - Twisted

[Twisted-Python] Thank You Twisted!
by Matthew Glubb June 26, 2007

June 26, 2007

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi All, I just wanted to say thanks for all of the good work with twisted. I have just finished testing my server application on a 1GB box for 1 hour with 10,000 simultaneous connections and it ran like a dream :) Granted, trying anything other than the epoll reactor brought the server to its knees very quickly. By the way, for any of you using memcache, don't be tempted by cmemcache if you are deferring those operations to a thread. Seems like cmemcache is not thread safe. I was getting lots of seg faults until I switched back to the pure python memcache module. Thanks again :) Matt m a t t h e w g l u b b ________________________________________________________________________ Z Group PLC Tel: +44 (0) 8700 111 173 Fax: +44 (0) 8707 051 393 Txt: +44 (0) 7800 140 877 Web: <http://www.zgroupplc.com/> This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. The opinions expressed in this mail are those of the author and do not necessarily represent the views of the company. If you have received this email in error please notify <service(a)zgroupplc.com> -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.1 (Darwin) iD8DBQFGgPJbyI6MkdKPngkRAlX5AJ9VsThIgIiT9FoEyPlYOtS5O8T9wwCeM9LN DLTVdulqFZ3pPS4TnHRvIX0= =PCMS -----END PGP SIGNATURE-----

1 0

Re: [Twisted-Python] Beginner question: integrating with CORBA
by Chris Laws June 25, 2007

June 25, 2007

> > ---------- Forwarded message ---------- > From: "Dave Kirby" <D.Kirby(a)tideway.com> > To: <twisted-python(a)twistedmatrix.com> > Date: Mon, 25 Jun 2007 12:03:46 +0100 > Subject: [Twisted-Python] Beginner question: integrating with CORBA > > I have just started looking at Twisted, and it looks really cool and > useful. I have read through much of the docs and the O'Reilly book, but I > still have a head full of questions. > > > > First some background. I am considering using Twisted to replace a > component in a network management suite. All the components communicate > through CORBA (omniORB). The component I want to replace gets CORBA calls > that requests data from servers on the network, and the component retrieves > the data using ssh. Currently it spawns /usr/bin/ssh and uses pexpect to > control the interaction. Typically we will have 20-40 simultaneous calls in > progress to different IP addresses, each on a separate thread and each > spawning a separate ssh client. > > > > I want to replace this with a component that uses Twisted instead. From > what I have read Twisted should be ideal for this – I can use the conch > library to do the ssh calls and eliminate the need for spawning dozens of > separate processes (something that has been a major cause of bugs). > > > > The first problem I have is how to integrate the omniORB event loop with > the reactor event loop. My initial thoughts are to have the omniORB event > loop in one thread and run the reactor loop in another thread. When a CORBA > call comes in it will use callFromThread() to post a request to the Twisted > thread that will create the connection and handle the ssh session. Is this > feasible? Any gotchas to watch out for? > > > > However I have also seen reference to using threadedselectreactor for > mixing Twisted and foreign even loops ( > http://bob.pythonmac.org/archives/2005/04/17/twisted-and-foreign-event-loop…), > but I have no idea how to use this with omniORB. Is this worth > investigating further, or should I stick with my original idea? > > > > Replacing CORBA with the Twisted PerspectiveBroker is not an option, since > it would require extensively rewriting a dozen or so components. > > > > I have plenty of other questions, but will save them for another time. > > > > Dave Kirby > > I just got back into a similar thing today after about a month of inactivity on the issue. I posted about it here a while back but got no responses. I'm trying to convert a simple Python/Omniorb CORBA Event Channel client script so that it can robustly handle a separate federated Name Service disappearing. I'm planning on running it using twisted for various reasons (common code, available functionality, etc). So I need to work out if I can use Twisted and CORBA (Omniorb to be specific) together, ideally within the same thread. Currently the simple script has a Tkinter UI which also registers a callback method with the Event Channel client wrapper to handle Event Messages. A self-rescheduling callback is also configured to drive the Orb, at some variable rate, and then the script is handled by the Tkiner mainloop. At regular intervals the Orb is run then rescheduled and then control is returned back to the Tk mainloop. I'm at home and can't remember the exact code but something along the lines of: def runOrb(self): if self.orb.work_pending(): self.orb.perform_work() This all seems to work well. However the script is fairly simple in that it's only monitoring and reacting to Event Channel messages, so slightly different from your use. However, I think that it demonstrates that the orb can be run from within another event processing loop within the same single thread - in this case the Tk mainloop. Now the Tk event processing can be handled by the tksupport package so I thought why not use that model for a CORBA support package? I've totally copied the tksupport file (it's very small) and converted it to perform the running and re-scheduling of the CORBA orb. Sorry I can't provide you more info but as mentioned above I only got back into this today (no testing yet) plus I finish work at that company in a few days. If I get anywhere I'll post back some info. Hopefully it gives you something to look into. Regards, Chris Laws

1 0

[Twisted-Python] Call to ffmpeg using getProcessOutput
by Daniel de la Cuesta June 25, 2007

June 25, 2007

2 2

[Twisted-Python] Beginner question: integrating with CORBA
by Dave Kirby June 25, 2007

June 25, 2007

I have just started looking at Twisted, and it looks really cool and useful. I have read through much of the docs and the O'Reilly book, but I still have a head full of questions. First some background. I am considering using Twisted to replace a component in a network management suite. All the components communicate through CORBA (omniORB). The component I want to replace gets CORBA calls that requests data from servers on the network, and the component retrieves the data using ssh. Currently it spawns /usr/bin/ssh and uses pexpect to control the interaction. Typically we will have 20-40 simultaneous calls in progress to different IP addresses, each on a separate thread and each spawning a separate ssh client. I want to replace this with a component that uses Twisted instead. From what I have read Twisted should be ideal for this - I can use the conch library to do the ssh calls and eliminate the need for spawning dozens of separate processes (something that has been a major cause of bugs). The first problem I have is how to integrate the omniORB event loop with the reactor event loop. My initial thoughts are to have the omniORB event loop in one thread and run the reactor loop in another thread. When a CORBA call comes in it will use callFromThread() to post a request to the Twisted thread that will create the connection and handle the ssh session. Is this feasible? Any gotchas to watch out for? However I have also seen reference to using threadedselectreactor for mixing Twisted and foreign even loops (http://bob.pythonmac.org/archives/2005/04/17/twisted-and-foreign-event- loops/), but I have no idea how to use this with omniORB. Is this worth investigating further, or should I stick with my original idea? Replacing CORBA with the Twisted PerspectiveBroker is not an option, since it would require extensively rewriting a dozen or so components. I have plenty of other questions, but will save them for another time. Dave Kirby This email and any attachment may contain confidential, privileged information for the sole use of the intended recipient. If you are not the intended recipient, do not disclose, reproduce, disseminate or otherwise use this communication. If you received this communication in error, please immediately notify the sender via email and delete the communication from your system. Company information : Tideway Systems Ltd, Registered Office: Anchor House, 15-19 Britten Street, London, SW3 3TY. Registered in England & Wales Reg. Reg. No: 4598072 VAT No: 805 5153 50

2 1

[Twisted-Python] Utility program to hex-dump network traffic
by Andrew E June 25, 2007

June 25, 2007

Dear Twisted Experts, About 2 years ago I needed a simple way to hexdump network traffic. I threw together a Twisted application, and have improved it little by little whenever I needed to. I didn't want to use an existing network sniffer tool. Instead, I borrowed the idea of a tunnel from ssh: Normally application A connects directly to application B; instead, A connects to the tunnel which connets to B. I posted the program to the ASPN site: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/502293 Though it works well for my purposes, I'm sure I've got some aspects wrong -- any/all comments/feedback/advice would be warmly appreciated :) Regards Andrew

2 2

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com June 24, 2007

June 24, 2007

Bug summary ______________________ Summary for 2007-06-17 through 2007-06-24 Bugs opened: 10 Bugs closed: 2 Total open bugs: 980 (+8) |== Type Changes |== Priority Changes |== Component Changes |Defect: +3 |Highest: +1 |Conch: +3 |Enhancement: +6 |High: +0 |Core: +4 |Normal: +8 |Lore: -1 |Trial: +1 |Web: +1 |Web2: +2 |Words: -1 New / Reopened Bugs ______________________ ===== Highest ===== [#2707] twisted.conch.ssh.connection has poor tests (opened by z3p) defect conch http://twistedmatrix.com/trac/ticket/2707 [#2709] twisted.conch.ssh.channel has poor tests (opened by z3p) defect conch http://twistedmatrix.com/trac/ticket/2709 ===== High ===== [#2710] twisted.conch.ssh.session has poor tests (opened by z3p) defect conch http://twistedmatrix.com/trac/ticket/2710 ===== Normal ===== [#2706] Add GNUTLS support in twisted (opened by yqiang) enhancement core http://twistedmatrix.com/trac/ticket/2706 [#2708] twisted.python.log.msg isn't documented in the api documentation (opened by exarkun) defect core http://twistedmatrix.com/trac/ticket/2708 [#2711] ConnectionRefusedError doesn't provide any information about the connection that was refused (opened by jml) enhancement core http://twistedmatrix.com/trac/ticket/2711 [#2712] cooperator should provide an API to pause, resume, and cancel individual tasks within the cooperator instead of the entire thing. (opened by glyph) enhancement core http://twistedmatrix.com/trac/ticket/2712 [#2713] trial -u can fail doctests on second pass (opened by mgeisler) defect trial http://twistedmatrix.com/trac/ticket/2713 [#2714] Add http_proxy and https_proxy support to web/client.py (opened by fsteinel) enhancement web http://twistedmatrix.com/trac/ticket/2714 [#1436] add getCookie/setCookie to web2.http_headers.Headers (opened by andrea) enhancement web2 http://twistedmatrix.com/trac/ticket/1436 Closed Bugs ______________________ ===== Highest ===== [#2701] twisted.words.xish.yappsrt.Scanner.__init__ is not proper epytext (opened by therve, closed by exarkun, duplicate) defect words http://twistedmatrix.com/trac/ticket/2701 ===== High ===== [#812 ] Lore always generates "Index" links even when meaningless, and even on index pages themselves. (opened by tirkal, closed by exarkun, fixed) defect lore http://twistedmatrix.com/trac/ticket/812

1 0

[Twisted-Python] protocol-factory question
by Luca Politti June 22, 2007

June 22, 2007

Hi group, I'm new in this mailing list and in twisted... I want to ask you a question about twisted factory and protocols: I'm building a server that have to receive commands from some clients via TCP. So I wrote my Protocol class, and my Factory class. If I understood the behavior of this system, every time a client connects to my server, factory build a protocol instance and "attach" it to the client. So the client comunicates with that protocol instance, right? Now, when from protocol I call self.transport.loseConnection(in way to disconnnect itself), the client will be disconnected,but its protocol instance still lives on. What can I do to destroy protocol instance after its client disconnection? Because if I have a thousand of connection attempts, I'll have a thousand of protocol instances, with a lot of allocated resources I hope I've been clear enough, even if my english is far to be perfect :) Thanks a lot Luca

2 1

[Twisted-Python] two twisted clients, how can I trigger the other one's callback in one's callback?
by slowtech chen June 21, 2007

June 21, 2007

I have 2 clients, one gets a message from a server, once I get the message, I want to send it with the other client to another server. how can i do this? can I add some customized events and notice the other client? Thanks. --------------------------------- Get the free Yahoo! toolbar and rest assured with the added security of spyware protection.

3 4

Re: [Twisted-Python] C++ port of Twisted
by Rodrigo Strauss June 20, 2007

June 20, 2007

Have tou tried Boost Asio? http://asio.sourceforge.net/ Rodrigo Strauss

2 1

[Twisted-Python] udp
by Bruce Coram June 20, 2007

June 20, 2007

How do you send data using udp? I want to use udp to send messages from clients to a server. I don't want to use connected udp. Can anyone supply some specimen code for me Thanks Bruce Coram

4 4