Mailman 3 May 2010 - Twisted

[Twisted-Python] Weekly Bug Summary
by exarkun＠twistedmatrix.com May 16, 2010

May 16, 2010

1 0

[Twisted-Python] Detecting idle connection with perspective broker
by Gabriel González May 14, 2010

May 14, 2010

Hi there, I wrote a small server application with PB but now I have some problem with client disconnection. When I have a idle connection, the logout method at server never is called (this method let me have a list of current logged users) and then client appears always connected to server. I can see the same issue with the chatserver example from perspective broker doc with detached method. class ChatRealm: implements(portal.IRealm) def requestAvatar(self, avatarID, mind, *interfaces): assert pb.IPerspective in interfaces avatar = User(avatarID) avatar.server = self.server avatar.attached(mind) return pb.IPerspective, avatar, lambda a=avatar:a.detached(mind) How can I detect that idle connection clients to call detached method? Thanks! -- Aquest missatge ha estat analitzat per MailScanner a la cerca de virus i d'altres continguts perillosos, i es considera que està net.

3 6

[Twisted-Python] suggestion for a file transfer protocol
by Gabriele Lanaro May 12, 2010

May 12, 2010

I'm trying to develop a simple application that let communicate two computers in a LAN for transferring files. My idea is that: using dbus/zeroconf each machine can see other's service and can connect with it (I need something without authentication or so) having a lan with 2 machines A asks B if he can send a file if B accepts, begin transferring if B refuse, send an error back It would be nice to transfer big files (not loaded all in memory) and the ability to stop/resume the upload/download and the integrity check. Is there a protocol that let me do this stuff or is better to implement a protocol on my own? (I'd like that the system is flexible to further extend for example, implement shared folders and so on) In the latter case it's convenient to use one port or two ports like ftp does?

3 3

[Twisted-Python] Sporadic GeneratorExit when using deferredGenerator
by CARSON DEAN FENIMORE May 12, 2010

May 12, 2010

I receive a GeneratorExit occasionally when using Twisted. Has anyone noticed this problem? t I am on python 2.5 - as I understand it, GeneratorExit derives from StandardError in 2.5, and BaseException in 2.6. The real problem in my code appears to be that the reference to my generator itself is being dropped, causing it to become invalidated. Additionally, the errback/callback chain is not propagating the error correctly. Has anyone experienced similar problems?

2 3

Re: [Twisted-Python] txriak module released.
by Ray Cote May 12, 2010

May 12, 2010

Hi Petr: Glad to hear you're finding the txriak module useful. Guess I need to make sure that all the listing pages are pointing to each other, and that they eventually point to the bitbucket sources. Look forward to hearing feedback. Thanks --Ray ----- Original Message ----- From: "Petr Mifek" <Petr.Mifek(a)anapol.cz> To: "Twisted general discussion" <twisted-python(a)twistedmatrix.com> Cc: "Ray Cote" <rgacote(a)AppropriateSolutions.com> Sent: Tuesday, May 11, 2010 4:01:14 PM GMT -05:00 US/Canada Eastern Subject: Re: [Twisted-Python] txriak module released. Ah, scratch that. Found the 0.2.1 update with bitbucket homepage set ;) Petr Mifek wrote: > Hi Ray, > > that is a great news! I'm already using it in a project. Did I miss a > wiki/page for it or so far none exists? Considered putting the source up > to http://github.com or similar site to allow for easy following the > code, extending & fixing issues? > > Cheers, Petr > > Ray Cote wrote: > >> Hello Twisted: >> >> An initial release of txriak has been posted to PyPI: >> <http://pypi.python.org/pypi/txriak/0.1.1> >> >> txriak is a Twisted module for communicating with Basho Technology's Riak data store via the HTTP interface. >> Written in Python, it requires Python 2.5 since it uses the yield version of deferreds. >> Initial release was written to match as closely as possible the 0.9 riak.py Python module released by Basho Technologies. >> >> Hope others find this to be useful. >> Look forward to comments and feedback. >> --R >> >> >> >> > > _______________________________________________ > Twisted-Python mailing list > Twisted-Python(a)twistedmatrix.com > http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python > -- Ray Cote, President Appropriate Solutions, Inc. We Build Software 603.924.6079

1 0

[Twisted-Python] txriak module released.
by Ray Cote May 12, 2010

May 12, 2010

Hello Twisted: An initial release of txriak has been posted to PyPI: <http://pypi.python.org/pypi/txriak/0.1.1> txriak is a Twisted module for communicating with Basho Technology's Riak data store via the HTTP interface. Written in Python, it requires Python 2.5 since it uses the yield version of deferreds. Initial release was written to match as closely as possible the 0.9 riak.py Python module released by Basho Technologies. Hope others find this to be useful. Look forward to comments and feedback. --R -- Ray Cote, President Appropriate Solutions, Inc. We Build Software 603.924.6079

3 3

[Twisted-Python] listenTCP in a TAC
by Donal McMullan May 11, 2010

May 11, 2010

I've got a server application that listens on a port, and when clients connect, spawns another TCP service on a random port that can live from a few seconds, up to a day or two. I had some problems getting this to work with twisted.application.internet.TCPServer (my services wouldn't stop listening when I'd finished with them) but it's working ok with plain old reactor.listenTCP. I don't really grok the TAC thing yet, or why there's a different method for spawning servers within an Application. Is my listenTCP approach bad practice for some reason that's not apparent to me? Thanks! Donal

3 4

Re: [Twisted-Python] Comparing "Stackless Python + Nonblocking Stackless Modules" with Twisted.
by Andrew Francis May 11, 2010

May 11, 2010

Hi Peter: Message: 7 Date: Sun, 9 May 2010 23:24:00 +0800 From: Peter Cai <newptcai(a)gmail.com> Subject: [Twisted-Python] Comparing "Stackless Python + Nonblocking Stackless Modules" with Twisted. To: Twisted general discussion <twisted-python(a)twistedmatrix.com> Message-ID: <o2ybefee6d81005090824l63d3f53fyc62a9d48f19093ac(a)mail.gmail.com> Content-Type: text/plain; charset=UTF-8 >Today, a guy gave me an URL >http://code.google.com/p/stacklessexamples/wiki/StacklessNonblockModules .... >With this module and stackless python, theoretically, we can build a >high concurrency network application framework which has a programming >style close to traditional multi-thread module. Drop the theoretical part. >The guy who gave me that URL asked a question, "Which style is better? >The Twited's event-driven style or the imaginary one?" >What do you think about it? Of course, twisted is an mature framework >we can trust, but doesn't the imaginary one also have pros? Well stackless python with an asynchronous networking package is not an imaginary style. Lots of people are using this to great effect (i.e., EVE-Online)- I'll argue the underlying design pattern is based on Doug Schmitt's half-sync half-async" pattern. The idea of the stacklesssocket module is to act as a plug in replacement for the various networking packages one may use. Most of the time, things work. However I feel there are a few gotchas that one has to be aware of. Myself, I use Stackless with Twisted. Outside of a hiccup in the later versions (I think I need to set recursion depth for some weird reason), I am happy. I like the one-stop big box shopping I get from using Twisted. That said, I think for simple network interactions - that is programmes that interact with a single end-point, Twisted is perfectly fine. For that matter, even if one interacts with multiple end-points in parallel, Twisted is fine - use a deferredlist. Most programmes fall into this category. I think things get more complex when one is dealing with multiple end-points in sequence (now callbacks are chained) or one is using iterators (cooperators/coiterators now needed?). What light-weight threads do is make things synchronous and make the control flow obvious to the programmer (because one is relying on Python to provide the control flow). For better or worse, I do stuff pertaining to process orchestration and it is just out there at times. Cheers, Andrew

3 2

[Twisted-Python] deferreds and unpacking binary data
by Xtroce May 11, 2010

May 11, 2010

Question: A Protocol should unpack some binary data. this is done in seperate calls to struct.unpack . Inbetween i do some checks for length, some strings returned are concatenated using join and logs are written. This all happens in the factory of the Protocol. To keep the code unblocking would it be wiser to spawn new Threads from the Protocol around the whole function, like: d = threads.deferToThread(self.factory.readPackage, data) because it all has to happen after one another anyway. Or should i split it up in deferreds which if i get it correctly would mean to create callback functions for each of the unpack()'s, join()'s and str()'s thanks

2 1

[Twisted-Python] inlineCallbacks vs explicit callbacks: performance concerns?
by Paul Goins May 10, 2010

May 10, 2010

Question: Is there a performance related reason why Twisted core does not use inlineCallbacks? ----- I'm trying to figure out how to reduce the CPU load of an app. I've already done a lot of profiling and have trimmed a lot of fat, but we're still not getting as much throughput as we'd like. We don't seem to really be I/O bound in our application, but mostly CPU bound. One thing we use a lot in our code is the inlineCallbacks decorator. It's also used by txAMQP, at least in the 0.2 version we are currently using. However, I notice it doesn't seem to really be used anywhere in the Twisted core code; I only see its definition and the defgen test which uses it. I'm curious of the reason why Twisted core does not use it. Is it performance related? Compatibility with older Python versions? Or something else? Thanks. - Paul Goins ---- For those curious, here's what I'm currently facing: If I count total "internal time" per module (NOT cumulative time), the current hotspot is twisted.internet.defer with 27.88% of all CPU time. The per-function breakdown is something like this: ======================= ===================== Line(Function) Time (total time pct) ======================= ===================== 314(_runCallbacks) 7.657 733(_inlineCallbacks) 6.246 833(unwindGenerator) 2.864 172(addCallbacks) 2.103 291(_startRunCallbacks) 1.940 206(addBoth) 1.369 766(gotResult) 1.165 233(callback) 1.014 ======================= ===================== Twisted version: 8.2.0. Profiler: cProfile. I'm trying to find the best way of getting this down. It seems like reducing the raw number of callbacks is probably the best way if possible. However, unwindGenerator/_inlineCallbacks by itself takes almost 10% of the time... hence the above question.

2 1