[Twisted-Python] threads

I'm trying to get Twisted 1.2.0 up on the OS/2 EMX port of Python I maintain. Most of it appears to be no great drama, but when I run the test suite, and it hits a threads related test things get awfully out of shape - the process goes into a runaway state that is unkillable, using all available CPU, although the system remains generally usable. A reboot is required to restore order. When I started the testing, I had an OS/2 port of SQLite+PySQLite installed, and the SQLite test triggered the above behaviour. On the assumption that my SQLite port could be implicated, I deinstalled SQLite etc and tried again. This uncovered a typo in twisted/flow/threads.py: *** threads.py.orig Sun Aug 10 13:00:48 2003 --- threads.py Mon Mar 15 17:12:16 2004 *************** *** 122,128 **** except StopIteration: reactor.callFromThread(self._stopping) except: ! self.failure = Faliure() reactor.callFromThread(self._cooperate) self._cooperate.immediate = True --- 122,128 ---- except StopIteration: reactor.callFromThread(self._stopping) except: ! self.failure = Failure() reactor.callFromThread(self._cooperate) self._cooperate.immediate = True With that fixed, I'm now back to the original behaviour. The test log contains the following: ... 2004/03/15 17:13 est [-] --> twisted.test.test_flow.FlowTest.testProtocolLocalhost <-- 2004/03/15 17:13 est [-] twisted.internet.protocol.ServerFactory starting on 0 2004/03/15 17:13 est [-] Starting factory <twisted.internet.protocol.ServerFactory instance at 0x2de70c> 2004/03/15 17:13 est [-] Starting factory <twisted.internet.protocol.ClientFactory instance at 0xb4780c> 2004/03/15 17:13 est [_Protocol,client] Stopping factory <twisted.internet.protocol.ClientFactory instance at 0xb4780c> 2004/03/15 17:13 est [-] --> twisted.test.test_flow.FlowTest.testQueryIterator <-- 2004/03/15 17:13 est [-] --> twisted.test.test_flow.FlowTest.testThreaded <-- 2004/03/15 17:13 est [-] Traceback (most recent call last): File "Lib/threading.py", line 436, in __bootstrap self.run() File "Lib/threading.py", line 416, in run self.__target(*self.__args, **self.__kwargs) --- <exception caught here> --- File "F:/dev/Twisted-1.2.0/twisted/python/threadpool.py", line 157, in _worker context.call(ctx, function, *args, **kwargs) File "F:/dev/Twisted-1.2.0/twisted/python/context.py", line 53, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "F:/dev/Twisted-1.2.0/twisted/python/context.py", line 32, in callWithContext return func(*args,**kw) File "F:/dev/Twisted-1.2.0/twisted/flow/threads.py", line 126, in _process reactor.callFromThread(self._cooperate) File "F:/dev/Twisted-1.2.0/twisted/internet/base.py", line 200, in callFromThread self.threadCallQueue.append((f, args, kw)) exceptions.AttributeError: 'NoneType' object has no attribute 'append' 2004/03/15 17:13 est [twisted.internet.protocol.Factory] (Port 0 Closed) 2004/03/15 17:13 est [twisted.internet.protocol.Factory] Stopping factory <twisted.internet.protocol.Factory instance at 0xaad8cc> 2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] (Port 0 Closed) 2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] Stopping factory <twisted.internet.protocol.ServerFactory instance at 0x2de70c> 2004/03/15 17:13 est [Foo,client] Stopping factory <twisted.internet.protocol.ClientFactory instance at 0xabc0ac> 2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] (Port 0 Closed) 2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] Stopping factory <twisted.internet.protocol.ServerFactory instance at 0xabc26c> 2004/03/15 17:13 est [Foo,client] Stopping factory <twisted.internet.protocol.ClientFactory instance at 0xabcb0c> 2004/03/15 17:13 est [Foo,client] Stopping factory <twisted.internet.protocol.ClientFactory instance at 0xabcd0c> 2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] (Port 0 Closed) 2004/03/15 17:13 est [twisted.internet.protocol.ServerFactory] Stopping factory <twisted.internet.protocol.ServerFactory instance at 0xabc14c> <<<log ends>>> The traceback is identical to that in the log after the SQLite test failure. While the threads support in the EMX port of Python passes the Python test suite, and people are successfully running Zope on OS/2 with this Python port, I don't know of any other widespread usage which might have exercised it. So I won't be surprised if it has problems. Can anyone give me any hints about what might be missing or working incorrectly to trigger the above? I should note that I've never tried to use Twisted before. The above is with Python 2.3.3. I don't have permanent internet connectivity, and the test was being run without an internet connection. Regards, Andrew. -- Andrew I MacIntyre "These thoughts are mine alone..." E-mail: andymac@bullseye.apana.org.au (pref) | Snail: PO Box 370 andymac@pcug.org.au (alt) | Belconnen ACT 2616 Web: http://www.andymac.org/ | Australia

On Wed, 2004-03-17 at 07:15, Andrew MacIntyre wrote:
except StopIteration: reactor.callFromThread(self._stopping) except: ! self.failure = Faliure() reactor.callFromThread(self._cooperate) self._cooperate.immediate = True
--- 122,128 ---- except StopIteration: reactor.callFromThread(self._stopping) except: ! self.failure = Failure() reactor.callFromThread(self._cooperate) self._cooperate.immediate = True
Yeesh. I fixed this, and attempted to make flow thread tests work. Try cvs and tell us if it's working now? (apparently no one did twisted.python.threadable.init(1) before using threads. Not sure why it works on linux though....). -- Itamar Shtull-Trauring http://itamarst.org Looking for a job: http://itamarst.org/resume.html
participants (2)
-
Andrew MacIntyre
-
Itamar Shtull-Trauring