Mailman 3 August 2005 - Twisted

[Twisted-Python] deferToThread and thread pool
by Brian Granger 31 Aug '05

31 Aug '05

I have a few questions about the default implementation of deferToThread. 1. Does deferToThread use a thread pool by default? If not, how can I enable this? 2. Which thread are the callbacks and errbacks of the deferred returned by deferToThread run in? Thanks in advance! Brian

3 2

[Twisted-Python] Nagle delays and twisted
by Brian Granger 30 Aug '05

30 Aug '05

Hello all, I have been implementing a custom protocol with twisted and have been having some odd performance problems. My protocol consists of a TCP connection with multiple client/server commands. For some types of commands, I found it took about 200 ms for the client and server to complete. Because this was over loopback (which has a ping of 0.1 ms), it seems very slow. Because I am keeping the socket connected, I am not measuring the socket setup time. Lucky for me, there was a discussion about this issue on comp.lang.python recently: http://groups.google.com/group/comp.lang.python/browse_frm/thread/ 507a711c61df7ad5/d6e871a24d8506a6#d6e871a24d8506a6 I guess the 200 ms delay is a well known effect of the Nagle Algorithm that TCP uses. Supposedly, this delay can be eliminated by i) setting TCP_NODELAY on the socket or ii) avoiding sending multiple small messages without a response from the other end. My understanding is that protocols that do the following will run into the problem: C: short message 1 C: short message 2 S: reply for message 1 and 2 What is the preferred way of setting TCP_NODELAY on a socket controlled by twisted? Are there any adverse side effects that this will have? And second: does anyone have any tips for designing protocols that avoid this delay without setting TCP_NODELAY? The obvious solution is to have a reply for each message: C: short message 1 S: reply for 1 C: short message 2 S: reply for 2 But... this increases the latency of the protocol. Any ideas or experiences with these issues would be greatly appreciated. Thanks Brian

2 1

[Twisted-Python] gthreadless.py
by Stefano Masini 30 Aug '05

30 Aug '05

Hello twisted folks, I reimplemented gthreadless.py, originally implemented by Chris Armstrong. My implementation aims to be more robust and is currently used in a large project, therefore it is already usable. http://www.pragma2000.com/wiki/index.php?GThreadless gthreadless is a twisted module that makes use of greenlets to implement the very nice primitive blockOn(d), that suspends the execution in the current stack frame while waiting for the deferred to fire. This way you can use a synchronous blocking-style programming model while still using twisted, all nicely integrated. It's a great great thing. Give it a try. cheers, Stefano Masini

5 10

[Twisted-Python] multiple connection to mysql with adbapi
by bahriesz＠yahoo.com 30 Aug '05

30 Aug '05

Hello, i tried to make a connection to mysql with adbapi and call thread that make use of these connection. Everytime i call the thread, it makes new connection to mysql. How do i close this connection in the thread once it finish? thanks. __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com

1 0

[Twisted-Python] Re: [Twisted-commits] r13641 - No, consigning copyright means it is _not_ (C) you. Let's keep this file simple, please.
by Mary Gardiner 30 Aug '05

30 Aug '05

On Mon, Apr 25, 2005, Glyph Lefkowitz wrote: > Modified: > trunk/LICENSE > Log: > No, consigning copyright means it is _not_ (C) you. Let's keep this file simple, please. [snip] > +Copyright (c) 2004,2005 Twisted Matrix Labs, http://www.twistedmatrix.com/ > +(except as listed below) > > Permission is hereby granted, free of charge, to any person obtaining > a copy of this software and associated documentation files (the > @@ -47,4 +23,5 @@ > Copyright Exceptions: > - > + - Portions copyright Massachusetts Institute of Technology, due to > + contributions of staff and students. I was reminded at the sprint that I needed to ask about this. My understanding of the license policy was as per http://twistedmatrix.com/bugs/issue903 -- ie, that the *only* thing commiters needed to do was to MIT license their contributions. This is all that http://twistedmatrix.com/developers/contributing currently states. Therefore, that was what I'd been getting people who contributed documentation to agree to, and that was my understanding of the licencing and copyright of my own contributions as of the date when the MIT licensing of Twisted began. If the current policy is that copyright must be consigned to an entity called "Twisted Matrix Labs", then this policy needs to be confirmed, restored to the webpage, and the status of all contributions at least since the website change and probably since the licence change needs to be confirmed with individual contributors. Also, my own network of nebulous legal advice from dubious sources suggests that Australian contributors might want to seek legal advice about how to do copyright consignment: it was suggested to me that the status of your moral right to be identified as author needs to be firmly nailed down for the consignment to have any validity whatsoever. -Mary

3 4

RE: [Twisted-Python] Use python logging module with Twisted log
by Henning.Ramm＠mediapro-gmbh.de 29 Aug '05

29 Aug '05

>The dictionary you get depends on whether it's log.msg or log.err. You >can then do what you want with it. A little bit of information, what keys can occurr and what they mean would be nice. How about this (if self.log is a logging.Logger); code is partly stolen from twisted.python.log: def twisted_logger(self, logdict): """ log observer for twisted.python.log, receives a dict and writes to self.log """ if 'failure' in logdict: text = logdict['failure'].getTraceback() else: text = " ".join([str(m) for m in logdict["message"]]) if logdict['isError']: self.log.error(text) else: self.log.warning(text) Best regards, Henning Hraban Ramm Südkurier Medienhaus / MediaPro Support/Admin/Development Dept.

1 0

[Twisted-Python] Use python logging module with Twisted log
by Remi Cool 29 Aug '05

29 Aug '05

G'day, I've created an xmlrpc server with twisted and was wondering if it would be possible to use twisted logging and the python logging module together. This because the xmlrpc server uses a 'back-end' that contains python modules that are also used by other processes/servers. It should be possible if a twisted handler for the python logging module existed ... it would then be a case of attaching the twisted handler to the logger and the connection would be established. Before I take on this task myself, I wonder if someone else has created such a handler or maybe has some other/better way of interconnecting to the python logging module? BTW .. where can I find some (detailed) documentation on the twisted logging system use? Regards, Remi Cool

2 1

RE: [Twisted-Python] Use python logging module with Twisted log
by Henning.Ramm＠mediapro-gmbh.de 29 Aug '05

29 Aug '05

>Before I take on this task myself, I wonder if someone else has created >such a handler or maybe has some other/better way of interconnecting to >the python logging module? Sorry, I can't help you, but I'd need the same (plus logging to some wx window), and I've no time to investigate and write it myself - so would you please release it, if you implement such? I'd be very glad! Best regards, Henning Hraban Ramm Südkurier Medienhaus / MediaPro Support/Admin/Development Dept.

1 0

[Twisted-Python] twisted 1.3.0 unit test errors
by Warren Pollans 29 Aug '05

29 Aug '05

Hello, I've started installing selenium on my ibook (OSX.3.9, python 2.3) I just installed Twisted 1.3.0. When I run admin/runtests from the install directory, I get the following results - Are these failures and errors acceptable? I've installed pyOpenSSL and pyCrypto, but not pySerial FAILED (failures=4, errors=2, skips=33, expectedFailures=4) Are these acceptable? Or what do I need to do to try to fix this before continuing the selenium install? Thanks, Warren ======================================================================== ======= FAILURE: testWhich (twisted.test.test_process.UtilTestCase) ------------------------------------------------------------------------ ------- Traceback (most recent call last): File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 252, in _runPhase stage(*args, **kwargs) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 274, in _main self.runner(self.method) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/runner.py", line 104, in runTest method() File "/Users/warren/Desktop/Twisted-1.3.0/twisted/test/test_process.py", line 661, in testWhich self.assertEquals(paths, [ File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 127, in failUnlessEqual raise FailTest, (msg or '%r != %r' % (first, second)) FailTest: ['foo/baz/executable', 'baz/foo/executable', 'baz/bar/executable'] != ['foo/baz/executable', 'baz/foo/executable'] ======================================================================== ======= FAILURE: testWhichPathExt (twisted.test.test_process.UtilTestCase) ------------------------------------------------------------------------ ------- Traceback (most recent call last): File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 252, in _runPhase stage(*args, **kwargs) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 274, in _main self.runner(self.method) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/runner.py", line 104, in runTest method() File "/Users/warren/Desktop/Twisted-1.3.0/twisted/test/test_process.py", line 676, in testWhichPathExt self.assertEquals(paths, [ File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 127, in failUnlessEqual raise FailTest, (msg or '%r != %r' % (first, second)) FailTest: ['foo/baz/executable', 'baz/foo/executable', 'baz/foo/executable.bin', 'baz/bar/executable'] != ['foo/baz/executable', 'baz/foo/executable', 'baz/foo/executable.bin'] ======================================================================== ======= FAILURE: testDownloadPageError3 (twisted.test.test_webclient.WebClientSSLTestCase) ------------------------------------------------------------------------ ------- Traceback (most recent call last): File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 252, in _runPhase stage(*args, **kwargs) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 274, in _main self.runner(self.method) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/runner.py", line 104, in runTest method() File "/Users/warren/Desktop/Twisted-1.3.0/twisted/test/test_webclient.py", line 175, in testDownloadPageError3 f = unittest.deferredError(d) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/util.py", line 106, in deferredError raise unittest.FailTest, "Deferred did not fail: %r" % (result,) FailTest: Deferred did not fail: None ======================================================================== ======= FAILURE: testDownloadPageError3 (twisted.test.test_webclient.WebClientTestCase) ------------------------------------------------------------------------ ------- Traceback (most recent call last): File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 252, in _runPhase stage(*args, **kwargs) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 274, in _main self.runner(self.method) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/runner.py", line 104, in runTest method() File "/Users/warren/Desktop/Twisted-1.3.0/twisted/test/test_webclient.py", line 175, in testDownloadPageError3 f = unittest.deferredError(d) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/util.py", line 106, in deferredError raise unittest.FailTest, "Deferred did not fail: %r" % (result,) FailTest: Deferred did not fail: None ======================================================================== ======= ERROR: testConnectionRefused (twisted.test.test_udp.OldConnectedUDPTestCase) ------------------------------------------------------------------------ ------- Traceback (most recent call last): File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 252, in _runPhase stage(*args, **kwargs) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 274, in _main self.runner(self.method) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/runner.py", line 104, in runTest method() File "/Users/warren/Desktop/Twisted-1.3.0/twisted/test/test_udp.py", line 123, in testConnectionRefused self.assertEquals(client.refused, 1) AttributeError: Client instance has no attribute 'refused' ======================================================================== ======= ERROR: testConnectionRefused (twisted.test.test_udp.UDPTestCase) ------------------------------------------------------------------------ ------- Traceback (most recent call last): File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 252, in _runPhase stage(*args, **kwargs) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/unittest.py", line 274, in _main self.runner(self.method) File "/Users/warren/Desktop/Twisted-1.3.0/twisted/trial/runner.py", line 104, in runTest method() File "/Users/warren/Desktop/Twisted-1.3.0/twisted/test/test_udp.py", line 216, in testConnectionRefused self.assertEquals(client.refused, 1) AttributeError: GoodClient instance has no attribute 'refused' ------------------------------------------------------------------------ ------- Ran 1227 tests in 501.362s FAILED (failures=4, errors=2, skips=33, expectedFailures=4)

2 1

[Twisted-Python] Interfaces and adapters
by en.karpachov＠ospaz.ru 28 Aug '05

28 Aug '05

Let's say we defined an interface and an adapter: class IFoo(zope.interfaces.Interface): pass twisted.python.components.registerAdapter(Foo, Bar, IFoo) As it is for now, the registerAdapter accepts any Foo, without regarding what it implements. Could somebody give any rationale why this condition shouldn't always hold: foo = IFoo(bar) assert IFoo.providedBy(foo) Seems it is not unreasonable to expect this behaviour, because it is exactly what interfaces are all about, or am I wrong? -- jk

6 15