[Twisted-Python] kqueue reactor / ticket #1918
Is there even a kqueue-based Twisted reactor?
Yes... ish. More detail here: http://twistedmatrix.com/trac/ticket/1918.
Hi Glyph,
first, thanks as always!
The pykqueue 1.3 and 2.0 referenced in that ticket seem to be no longer maintained.
On the other hand, Python has built-in support for kqueue since Python 2.6 in the select module.
I have adjusted the Twisted kqreactor to select.kqueue and made a package:
https://github.com/oberstet/txkqreactor
I have tested that reactor successfully with Autobahn WebSockets on FreeBSD 8.2 (running as a VirtualBox x86 VM). When I'm home later, I can check out Mac.
Doing the Autobahn tests helped me finding the only non-canonical change required
https://github.com/oberstet/txkqreactor/commit/27302b4f643cd3967289ff11d8113...
Is there any regression/test/stresstest I could run to verify the stuff actually works?
The Autobahn WS tests do quite some stuff, but they do not test concurrency and high connections numbers ..
Cheers, Tobias
On 02:11 pm, tobias.oberstein@tavendo.de wrote:
Is there even a kqueue-based Twisted reactor?
Yes... ish. �More detail here: http://twistedmatrix.com/trac/ticket/1918.
Hi Glyph,
first, thanks as always!
The pykqueue 1.3 and 2.0 referenced in that ticket seem to be no longer maintained.
On the other hand, Python has built-in support for kqueue since Python 2.6 in the select module.
I have adjusted the Twisted kqreactor to select.kqueue and made a package:
https://github.com/oberstet/txkqreactor
I have tested that reactor successfully with Autobahn WebSockets on FreeBSD 8.2 (running as a VirtualBox x86 VM). When I'm home later, I can check out Mac.
Doing the Autobahn tests helped me finding the only non-canonical change required
https://github.com/oberstet/txkqreactor/commit/27302b4f643cd3967289ff11d8113...
Is there any regression/test/stresstest I could run to verify the stuff actually works?
You can run the Twisted test suite:
trial --reactor kqueue twisted
(assuming you've hooked up the name "kqueue" to your new version of the kqueue reactor).
This is what needs to work in order for the new reactor to be added to Twisted itself (ie, for us to resolve #1918).
Jean-Paul
The Autobahn WS tests do quite some stuff, but they do not test concurrency and high connections numbers ..
Cheers, Tobias
Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
Is there any regression/test/stresstest I could run to verify the stuff actually works?
You can run the Twisted test suite:
trial --reactor kqueue twisted(assuming you've hooked up the name "kqueue" to your new version of the kqueue reactor).
This is what needs to work in order for the new reactor to be added to Twisted itself (ie, for us to resolve #1918).
Hello Jean-Paul,
I've ran the new one on a Python 2.7.1/Twisted 11.0 (stock) by patching
twisted_reactor.py
#kqueue = Reactor( # 'kqueue', 'twisted.internet.kqreactor', 'kqueue(2)-based reactor.') kqueue = Reactor( 'kqueue', 'txkqreactor.kqreactor', 'NEW kqueue(2)-based reactor.')
and running both that, and the select() reactor.
Uploaded all logs to the ticket.
The diff between both:
http://twistedmatrix.com/trac/attachment/ticket/1918/select_vs_new_kqueue_di...
Summary:
< Ran 6515 tests in 102.948s ---
Ran 6515 tests in 343.371s
9292c9354 < FAILED (skips=977, expectedFailures=11, failures=1, errors=7, successes=5519) ---
FAILED (skips=977, expectedFailures=11, failures=2, errors=13, successes=5516)
< = select
= new kqueue
Is that good, bad or ugly?
On Fri, Nov 4, 2011 at 9:11 AM, Tobias Oberstein < tobias.oberstein@tavendo.de> wrote:
Is there even a kqueue-based Twisted reactor?
Yes... ish. More detail here: <
http://twistedmatrix.com/trac/ticket/1918%3E.
Hi Glyph,
first, thanks as always!
The pykqueue 1.3 and 2.0 referenced in that ticket seem to be no longer maintained.
On the other hand, Python has built-in support for kqueue since Python 2.6 in the select module.
I have adjusted the Twisted kqreactor to select.kqueue and made a package:
https://github.com/oberstet/txkqreactor
I have tested that reactor successfully with Autobahn WebSockets on FreeBSD 8.2 (running as a VirtualBox x86 VM). When I'm home later, I can check out Mac.
Doing the Autobahn tests helped me finding the only non-canonical change required
https://github.com/oberstet/txkqreactor/commit/27302b4f643cd3967289ff11d8113...
Is there any regression/test/stresstest I could run to verify the stuff actually works?
The Autobahn WS tests do quite some stuff, but they do not test concurrency and high connections numbers ..
Cheers, Tobias
This is awesome, BUT...
...would probably have to be distributed outside of Twisted proper (at least for the moment), as Twisted supports versions of Python < 2.6, where select.kqueue would not exist.
(Maybe this is your plan, but I'm just pointing it out.)
Kevin Horn
This is awesome, BUT...
...would probably have to be distributed outside of Twisted proper (at least for the moment), as Twisted supports versions of Python < 2.6, where select.kqueue would not exist.
(Maybe this is your plan, but I'm just pointing it out.)
We've said e.g. Windows only supports Python 2.5 or later even when everything else supports Python 2.4. And we have inlineCallbacks which only runs on Python 2.5 or later. Saying "kqueue reactor only works on Python 2.6 and later" seems fine to me.
-Itamar
On 04:49 pm, kevin.horn@gmail.com wrote:
On Fri, Nov 4, 2011 at 9:11 AM, Tobias Oberstein < tobias.oberstein@tavendo.de> wrote:
Is there even a kqueue-based Twisted reactor?
Yes... ish. More detail here: <
http://twistedmatrix.com/trac/ticket/1918%3E.
Hi Glyph,
first, thanks as always!
The pykqueue 1.3 and 2.0 referenced in that ticket seem to be no longer maintained.
On the other hand, Python has built-in support for kqueue since Python 2.6 in the select module.
I have adjusted the Twisted kqreactor to select.kqueue and made a package:
https://github.com/oberstet/txkqreactor
I have tested that reactor successfully with Autobahn WebSockets on FreeBSD 8.2 (running as a VirtualBox x86 VM). When I'm home later, I can check out Mac.
Doing the Autobahn tests helped me finding the only non-canonical change required
https://github.com/oberstet/txkqreactor/commit/27302b4f643cd3967289ff11d8113...
Is there any regression/test/stresstest I could run to verify the stuff actually works?
The Autobahn WS tests do quite some stuff, but they do not test concurrency and high connections numbers ..
Cheers, Tobias
This is awesome, BUT...
...would probably have to be distributed outside of Twisted proper (at least for the moment), as Twisted supports versions of Python < 2.6, where select.kqueue would not exist.
(Maybe this is your plan, but I'm just pointing it out.)
It would be fine for Twisted to offer this feature only on Python 2.6 and newer.
Jean-Paul
This is awesome, BUT...
...would probably have to be distributed outside of Twisted proper (at least for the moment), as Twisted supports versions of Python < 2.6, where select.kqueue would not exist.
(Maybe this is your plan, but I'm just pointing it out.)
It would be fine for Twisted to offer this feature only on Python 2.6 and newer.
Would be great of course to have it in Twisted.
In the meantime, I've packaged it up
http://pypi.python.org/pypi/txkqreactor
Caveat: as attached to the ticket, there are some errs when running trial which don't happen with select reactor.
Whether those are significant I can't tell .. if there is anything I can do to clarify (like i.e. running on non-virtualized hardware FreeBSD) let me know.
On Nov 4, 2011, at 1:11 PM, Tobias Oberstein wrote:
This is awesome, BUT...
...would probably have to be distributed outside of Twisted proper (at least for the moment), as Twisted supports versions of Python < 2.6, where select.kqueue would not exist.
(Maybe this is your plan, but I'm just pointing it out.)
It would be fine for Twisted to offer this feature only on Python 2.6 and newer.
Would be great of course to have it in Twisted.
Yes :).
In the meantime, I've packaged it up
Thanks, but we should really try to integrate it ASAP. :)
Caveat: as attached to the ticket, there are some errs when running trial which don't happen with select reactor.
Do you think you could start fixing some of those errors? Do they make sense to you? If you're interested in moving this ticket forward, I'm sure that folks on this mailing list (me included) would love to provide you some guidance.
Whether those are significant I can't tell ..
If they're failing tests, they're significant :). In order for stuff to be added to Twisted, the full test suite needs to run.
if there is anything I can do to clarify (like i.e. running on non-virtualized hardware FreeBSD) let me know.
Have you run the tests on OS X as well?
-glyph
if there is anything I can do to clarify (like i.e. running on non-virtualized
hardware FreeBSD) let me know.
Have you run the tests on OS X as well?
I'll do that.
Caveat: as attached to the ticket, there are some errs when running trial which don't happen with select reactor.
Do you think you could start fixing some of those errors? Do they make
I've looked at the logs in more detail, the error _diff_ between select/kqueue yields:
twisted.test.test_tcp.HalfCloseBuggyApplicationTests.test_readNotificationRaises twisted.test.test_tcp.HalfCloseTestCase.testCloseWriteCloser twisted.test.test_tcp.HalfCloseTestCase.testWriteCloseNotification
which all seem to be testing TCP Half-Close.
I've looked at the case code .. guess I can grasp that after reading some background.
But I have no clue where to start "fixing" / looking for why this works with the select reactor vs kqueue.
sense to you? If you're interested in moving this ticket forward, I'm sure that folks on this mailing list (me included) would love to provide you some guidance.
Ok, will see what I can do.
Need some advice/hints though.
Generally, there are a couple of skipped tests missing modules (i.e. pycrypto) which I could install. Should I do that, or are those not necessary? Note, that there are packages I could, but would not like to install: GTK etc.
Then, whats the difference between FAIL and ERROR?
Then, there are a couple of tests which are skipped because
xxx_KQueueReactor No module named kqsyscall
The kqsyscall should now be select.kqueue/kevent .. so I guess those test cases needed to be adapted as well, right?
Then, baseline. I've run the trial using select reactor in the expectation of finding all tests passed.
This is not the case, there are a couple of failed ones. Those are also failed with the kqueue reactor, so I need to know whether I need to inspect those or those are problems unrelated to the reactor, but of general (platform?) nature.
[1] twisted.internet.test.test_posixprocess FileDescriptorTests test_expectedFDs ... [FAIL]
[2] twisted.internet.test.test_process PTYProcessTestsBuilder_PollReactor test_openFileDescriptors ... [ERROR] PTYProcessTestsBuilder_SelectReactor test_openFileDescriptors ... [ERROR] ProcessTestsBuilder_PollReactor test_openFileDescriptors ... [ERROR] ProcessTestsBuilder_SelectReactor test_openFileDescriptors ... [ERROR]
[3] twisted.python.test.test_components RegistrationTestCase test_duplicateAdapterForInterfaceAllowed ... [ERROR] test_registerAdapterForInterface ... [ERROR] test_subclassAdapterRegistrationForInterface ... [ERROR]
Now, [2] could be a consequence of [3], since they all fail because of
exceptions.ValueError: an adapter (<function <lambda> at 0x2c0dc41c>) was already registered.
See:
twisted.internet.test.test_process.PTYProcessTestsBuilder_PollReactor.test_openFileDescriptors twisted.internet.test.test_process.PTYProcessTestsBuilder_SelectReactor.test_openFileDescriptors twisted.internet.test.test_process.ProcessTestsBuilder_PollReactor.test_openFileDescriptors twisted.internet.test.test_process.ProcessTestsBuilder_SelectReactor.test_openFileDescriptors =============================================================================== [ERROR] Traceback (most recent call last): File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 351, in test_duplicateAdapterForInterfaceAllowed return self._duplicateAdapterForClassOrInterfaceAllowed(TheOriginal) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 312, in _duplicateAdapterForClassOrInterfaceAllowed components.registerAdapter(firstAdapter, original, TheInterface) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/components.py", line 93, in registerAdapter raise ValueError("an adapter (%s) was already registered." % (factory, )) exceptions.ValueError: an adapter (<function <lambda> at 0x2c0dc41c>) was already registered.
twisted.python.test.test_components.RegistrationTestCase.test_duplicateAdapterForInterfaceAllowed =============================================================================== [ERROR] Traceback (most recent call last): File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 268, in test_registerAdapterForInterface return self._registerAdapterForClassOrInterface(TheOriginal) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 245, in _registerAdapterForClassOrInterface components.registerAdapter(adapter, original, TheInterface) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/components.py", line 93, in registerAdapter raise ValueError("an adapter (%s) was already registered." % (factory, )) exceptions.ValueError: an adapter (<function <lambda> at 0x2c0dc41c>) was already registered.
twisted.python.test.test_components.RegistrationTestCase.test_registerAdapterForInterface =============================================================================== [ERROR] Traceback (most recent call last): File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 423, in test_subclassAdapterRegistrationForInterface return self._subclassAdapterRegistrationForClassOrInterface(TheOriginal) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/test/test_components.py", line 396, in _subclassAdapterRegistrationForClassOrInterface components.registerAdapter(firstAdapter, original, TheInterface) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/components.py", line 93, in registerAdapter raise ValueError("an adapter (%s) was already registered." % (factory, )) exceptions.ValueError: an adapter (<function <lambda> at 0x2c0dc41c>) was already registered.
twisted.python.test.test_components.RegistrationTestCase.test_subclassAdapterRegistrationForInterface
=======
And [1]:
twisted.internet.test.test_posixprocess.FileDescriptorTests.test_expectedFDs =============================================================================== [ERROR] Traceback (most recent call last): File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/_baseprocess.py", line 60, in maybeCallProcessEnded proto.processEnded(Failure(reason)) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/test/test_process.py", line 289, in processEnded checkOutput("".join(self.output)) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/test/test_process.py", line 276, in checkOutput self.assertEquals('[0, 1, 2, 3]', output) twisted.trial.unittest.FailTest: not equal: a = '[0, 1, 2, 3]' b = 'xrange(256)'
**********************************************
Now, for the kqeue reactor, the following _additional_ errors appear:
twisted.test.test_tcp HalfCloseBuggyApplicationTests test_readNotificationRaises ... [FAIL] HalfCloseTestCase testCloseWriteCloser ... [ERROR] testWriteCloseNotification ... [ERROR]
twisted.test.test_tcp.HalfCloseBuggyApplicationTests.test_readNotificationRaises =============================================================================== [ERROR] Traceback (most recent call last): File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/_baseprocess.py", line 60, in maybeCallProcessEnded proto.processEnded(Failure(reason)) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/test/test_process.py", line 289, in processEnded checkOutput("".join(self.output)) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/test/test_process.py", line 276, in checkOutput self.assertEquals('[0, 1, 2, 3]', output) twisted.trial.unittest.FailTest: not equal: a = '[0, 1, 2, 3]' b = 'xrange(256)'
=============================================================================== [ERROR] Traceback (most recent call last): Failure: twisted.internet.defer.TimeoutError: <twisted.test.test_tcp.HalfCloseTestCase testMethod=testCloseWriteCloser> (testCloseWriteCloser) still running at 120.0 secs
twisted.test.test_tcp.HalfCloseTestCase.testCloseWriteCloser =============================================================================== [ERROR] Traceback (most recent call last): Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean. DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug) <DelayedCall 0x29c0462c [-6.91413879395e-05s] called=0 cancelled=0 LoopingCall<0>(check, *(), **{})()>
twisted.test.test_tcp.HalfCloseTestCase.testCloseWriteCloser =============================================================================== [ERROR] Traceback (most recent call last): Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean. Selectables: <<class 'twisted.internet.tcp.Port'> of twisted.test.test_tcp.MyHCFactory on 53746>
twisted.test.test_tcp.HalfCloseTestCase.testCloseWriteCloser =============================================================================== [ERROR] Traceback (most recent call last): Failure: twisted.internet.defer.TimeoutError: <twisted.test.test_tcp.HalfCloseTestCase testMethod=testWriteCloseNotification> (testWriteCloseNotification) still running at 120.0 secs
twisted.test.test_tcp.HalfCloseTestCase.testWriteCloseNotification =============================================================================== [ERROR] Traceback (most recent call last): File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/test/test_tcp.py", line 1602, in tearDown self.assertEquals(self.client.closed, 0) twisted.trial.unittest.FailTest: not equal: a = 1 b = 0
twisted.test.test_tcp.HalfCloseTestCase.testWriteCloseNotification =============================================================================== [ERROR] Traceback (most recent call last): Failure: twisted.trial.util.DirtyReactorAggregateError: Reactor was unclean. DelayedCalls: (set twisted.internet.base.DelayedCall.debug = True to debug) <DelayedCall 0x2bf7f4ac [-6.29425048828e-05s] called=0 cancelled=0 LoopingCall<0>(check, *(), **{})()>
twisted.test.test_tcp.HalfCloseTestCase.testWriteCloseNotification
We've said e.g. Windows only supports Python 2.5 or later even when everything else supports Python 2.4. And we have inlineCallbacks which only runs on Python 2.5 or later. Saying "kqueue reactor only works on Python 2.6 and later" seems fine to me.
For completeness: there seem to be 2 packages providing the select.kqueue/epoll interface backported for Python 2.3-2.5:
http://pypi.python.org/pypi/select26/0.1a3 http://pypi.python.org/pypi/select_backport/0.2
However, both seem to be based on Python svn at some point in time, and not actively maintained.
On Sat, 2011-11-05 at 04:48 -0700, Tobias Oberstein wrote:
Then, whats the difference between FAIL and ERROR?
Fail means an assertion failed. Error means an unexpected exception was throwhn.
Then, there are a couple of tests which are skipped because
xxx_KQueueReactor No module named kqsyscall
The kqsyscall should now be select.kqueue/kevent .. so I guess those test cases needed to be adapted as well, right?
Yes.
Then, baseline. I've run the trial using select reactor in the expectation of finding all tests passed.
This is not the case, there are a couple of failed ones. Those are also failed with the kqueue reactor, so I need to know whether I need to inspect those or those are problems unrelated to the reactor, but of general (platform?) nature.
Are you using Twisted 11? Many of these issues are, if I'm not mistaken, fixed in trunk (though perhaps not all). So trunk is probably a better baseline to work off of.
Then, baseline. I've run the trial using select reactor in the expectation of finding all tests passed.
This is not the case, there are a couple of failed ones. Those are also failed with the kqueue reactor, so I need to know whether I need to inspect those or those are problems unrelated to the reactor, but of
general (platform?) nature.
Are you using Twisted 11? Many of these issues are, if I'm not mistaken, fixed in trunk (though perhaps not all). So trunk is probably a better baseline to work off of.
Ok, I'm using trunk now.
1) The errors in
twisted.python.test.test_components
are gone.
2) The errors in
twisted.internet.test.test_posixprocess twisted.internet.test.test_process
are still there ... and they show up for select, poll and kqueue reactors.
They all somehow relate to "open FDs":
twisted.internet.test.test_posixprocess FileDescriptorTests test_expectedFDs ... [FAIL]
twisted.internet.test.test_process PTYProcessTestsBuilder_KQueueReactor test_openFileDescriptors ... [ERROR] PTYProcessTestsBuilder_PollReactor test_openFileDescriptors ... [ERROR] PTYProcessTestsBuilder_SelectReactor test_openFileDescriptors ... [ERROR] ProcessTestsBuilder_SelectReactor test_openFileDescriptors ... [ERROR]
This might more a FreeBSD related thing. ?
3) I think I've nailed down the Half-Close stuff.
4) These are kqueue specific. Need to investigate further.
twisted.internet.test.test_fdset ReactorFDSetTestsBuilder_KQueueReactor test_lostFileDescriptor ... [ERROR] test_removedFromReactor ... [FAIL]
The first one comes up, since
removeReader()
is called for a FD which is gone, and leads to a socket exception from fd = reader.fileno()
5) These are kqueue specific. Need to investigate further.
twisted.conch.test.test_conch
The fail on the kqeue control call .. see below.
Any ideas what could be the reason?
[autobahn@autobahnhub ~/temp]$ trial -r select twisted.conch.test.test_conch twisted.conch.test.test_conch CmdLineClientTestCase test_exec ... [OK] test_localToRemoteForwarding ... [OK] test_remoteToLocalForwarding ... [OK] OpenSSHClientForwardingTestCase test_exec ... [OK] test_localToRemoteForwarding ... [OK] test_remoteToLocalForwarding ... [OK] OpenSSHClientRekeyTestCase test_clientRekey ... [OK] StdioInteractingSessionTests test_eofReceived ... [OK]
------------------------------------------------------------------------------- Ran 8 tests in 4.707s
PASSED (successes=8) [autobahn@autobahnhub ~/temp]$ trial -r kqueue twisted.conch.test.test_conch twisted.conch.test.test_conch CmdLineClientTestCase test_exec ... [OK] test_localToRemoteForwarding ... [ERROR] [ERROR] test_remoteToLocalForwarding ... [OK] OpenSSHClientForwardingTestCase test_exec ... [OK] test_localToRemoteForwarding ... [ERROR] [ERROR] test_remoteToLocalForwarding ... [OK] OpenSSHClientRekeyTestCase test_clientRekey ... [OK] StdioInteractingSessionTests test_eofReceived ... [OK]
=============================================================================== [ERROR] Traceback (most recent call last): Failure: twisted.internet.defer.TimeoutError: <twisted.conch.test.test_conch.CmdLineClientTestCase testMethod=test_localToRemoteForwarding> (test_localToRemoteForwarding) still running at 120.0 secs
twisted.conch.test.test_conch.CmdLineClientTestCase.test_localToRemoteForwarding =============================================================================== [ERROR] Traceback (most recent call last): File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/log.py", line 84, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/log.py", line 69, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/context.py", line 118, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/context.py", line 81, in callWithContext return func(*args,**kw) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/kqreactor.py", line 225, in _doWriteOrRead self._disconnectSelectable(selectable, why, inRead) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/posixbase.py", line 260, in _disconnectSelectable selectable.readConnectionLost(f) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/tcp.py", line 257, in readConnectionLost self.connectionLost(reason) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/tcp.py", line 433, in connectionLost Connection.connectionLost(self, reason) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/tcp.py", line 277, in connectionLost protocol.connectionLost(reason) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/conch/test/test_conch.py", line 242, in connectionLost self.protocol.forwardingPortDisconnected(self.buffer) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/conch/test/test_conch.py", line 190, in forwardingPortDisconnected self.transport.write('\x03') File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/process.py", line 862, in write self.pipes[0].write(data) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/process.py", line 174, in write abstract.FileDescriptor.write(self, data) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/abstract.py", line 310, in write self.startWriting() File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/abstract.py", line 400, in startWriting self.reactor.addWriter(self) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/kqreactor.py", line 94, in addWriter self._updateRegistration(fd, KQ_FILTER_WRITE, KQ_EV_ADD) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/kqreactor.py", line 74, in _updateRegistration self._kq.control([kevent(*args)], 0, 0) exceptions.OSError: [Errno 32] Broken pipe
twisted.conch.test.test_conch.CmdLineClientTestCase.test_localToRemoteForwarding =============================================================================== [ERROR] Traceback (most recent call last): Failure: twisted.internet.defer.TimeoutError: <twisted.conch.test.test_conch.OpenSSHClientForwardingTestCase testMethod=test_localToRemoteForwarding> (test_localToRemoteForwarding) still running at 120.0 secs
twisted.conch.test.test_conch.OpenSSHClientForwardingTestCase.test_localToRemoteForwarding =============================================================================== [ERROR] Traceback (most recent call last): File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/log.py", line 84, in callWithLogger return callWithContext({"system": lp}, func, *args, **kw) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/log.py", line 69, in callWithContext return context.call({ILogContext: newCtx}, func, *args, **kw) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/context.py", line 118, in callWithContext return self.currentContext().callWithContext(ctx, func, *args, **kw) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/python/context.py", line 81, in callWithContext return func(*args,**kw) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/kqreactor.py", line 225, in _doWriteOrRead self._disconnectSelectable(selectable, why, inRead) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/posixbase.py", line 262, in _disconnectSelectable self.removeWriter(selectable) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/kqreactor.py", line 140, in removeWriter self._updateRegistration(fd, KQ_FILTER_WRITE, KQ_EV_DELETE) File "/home/autobahn/python/lib/python2.7/site-packages/Twisted-11.0.0_r33077-py2.7-freebsd-8.2-RELEASE-p3-i386.egg/twisted/internet/kqreactor.py", line 74, in _updateRegistration self._kq.control([kevent(*args)], 0, 0) exceptions.OSError: [Errno 2] No such file or directory
twisted.conch.test.test_conch.OpenSSHClientForwardingTestCase.test_localToRemoteForwarding ------------------------------------------------------------------------------- Ran 8 tests in 243.797s
FAILED (errors=4, successes=6) [autobahn@autobahnhub ~/temp]$
On Nov 5, 2011, at 1:46 PM, Tobias Oberstein wrote:
Then, baseline. I've run the trial using select reactor in the expectation of finding all tests passed.
This is not the case, there are a couple of failed ones. Those are also failed with the kqueue reactor, so I need to know whether I need to inspect those or those are problems unrelated to the reactor, but of
general (platform?) nature.
Are you using Twisted 11? Many of these issues are, if I'm not mistaken, fixed in trunk (though perhaps not all). So trunk is probably a better baseline to work off of.
Ok, I'm using trunk now.
The errors in
twisted.python.test.test_components
are gone.
The errors in
twisted.internet.test.test_posixprocess twisted.internet.test.test_process
are still there ... and they show up for select, poll and kqueue reactors.
They all somehow relate to "open FDs":
twisted.internet.test.test_posixprocess FileDescriptorTests test_expectedFDs ... [FAIL]
twisted.internet.test.test_process PTYProcessTestsBuilder_KQueueReactor test_openFileDescriptors ... [ERROR] PTYProcessTestsBuilder_PollReactor test_openFileDescriptors ... [ERROR] PTYProcessTestsBuilder_SelectReactor test_openFileDescriptors ... [ERROR] ProcessTestsBuilder_SelectReactor test_openFileDescriptors ... [ERROR]
This might more a FreeBSD related thing. ?
Yes.
These tests should pass; I am guessing this has something to do with the still-open ticket http://tm.tl/4747. Maybe have a look at fixing that?
You might be able to work around it on your system this by mounting fdescfs at /dev/fd. (This really ought to be the default, please bother someone in FreeBSD-land to make it so. :)).
It would be useful to include tracebacks with the errors. "trial --rterrors" is useful, as it includes the tracebacks especially if you are experiencing hanging with the tests.
I think I've nailed down the Half-Close stuff.
Awesome. Do you have a link to a changeset for that? (Did I miss it earlier in the thread?)
These are kqueue specific. Need to investigate further.
These are the ones to focus on for http://tm.tl/1918, then :).
twisted.internet.test.test_fdset ReactorFDSetTestsBuilder_KQueueReactor test_lostFileDescriptor ... [ERROR] test_removedFromReactor ... [FAIL]
The first one comes up, since
removeReader()
is called for a FD which is gone, and leads to a socket exception from fd = reader.fileno()
It sounds like you've narrowed this down pretty far! Good work.
These are kqueue specific. Need to investigate further.
twisted.conch.test.test_conch
The fail on the kqeue control call .. see below.
Any ideas what could be the reason?
I would suggest fixing the issues you outlined in point 4 first. It's really unfortunate that reactor issues are being caught by conch tests. The reactor tests should be comprehensive enough to catch everything in the reactor.
This may just be a side-effect of the other errors you're tracking down, so see if you can fix those, and then loop back to these. If they disappear, then great. If not, then we should have a test that specifically tests this functionality and doesn't rely on weird and difficult to diagnose errors in conch to find them.
Thanks again for working on this!
-glyph
Ok, I've prepared
http://twistedmatrix.com/trac/attachment/ticket/1918/kqueue.patch
which passes all tests (FreeBSD 8.3 p3 i386 / Python 2.7.1) but the ones related to open FDs (which are unrelated to the reactor stuff).
This works for me for FreeBSD .. I've tried on MacOSX .. there seem to be PTY issues there. Well.
I've also updated the
http://pypi.python.org/pypi/txkqreactor/0.2
for anyone who wants to try before this gets eventually merged into Twisted.
Von: twisted-python-bounces@twistedmatrix.com [mailto:twisted-python-bounces@twistedmatrix.com] Im Auftrag von Glyph Lefkowitz Gesendet: Sonntag, 6. November 2011 22:13 An: Twisted general discussion Betreff: Re: [Twisted-Python] kqueue reactor / ticket #1918
On Nov 5, 2011, at 1:46 PM, Tobias Oberstein wrote:
Then, baseline. I've run the trial using select reactor in the expectation of finding all tests passed.
This is not the case, there are a couple of failed ones. Those are also failed with the kqueue reactor, so I need to know whether I need to inspect those or those are problems unrelated to the reactor, but of general (platform?) nature.
Are you using Twisted 11? Many of these issues are, if I'm not mistaken, fixed in trunk (though perhaps not all). So trunk is probably a better baseline to work off of.
Ok, I'm using trunk now.
1) The errors in
twisted.python.test.test_components
are gone.
2) The errors in
twisted.internet.test.test_posixprocess twisted.internet.test.test_process
are still there ... and they show up for select, poll and kqueue reactors.
They all somehow relate to "open FDs":
twisted.internet.test.test_posixprocess FileDescriptorTests test_expectedFDs ... [FAIL]
twisted.internet.test.test_process PTYProcessTestsBuilder_KQueueReactor test_openFileDescriptors ... [ERROR] PTYProcessTestsBuilder_PollReactor test_openFileDescriptors ... [ERROR] PTYProcessTestsBuilder_SelectReactor test_openFileDescriptors ... [ERROR] ProcessTestsBuilder_SelectReactor test_openFileDescriptors ... [ERROR]
This might more a FreeBSD related thing. ?
Yes.
These tests should pass; I am guessing this has something to do with the still-open ticket http://tm.tl/4747. Maybe have a look at fixing that?
You might be able to work around it on your system this by mounting fdescfs at /dev/fd. (This really ought to be the default, please bother someone in FreeBSD-land to make it so. :)).
It would be useful to include tracebacks with the errors. "trial --rterrors" is useful, as it includes the tracebacks especially if you are experiencing hanging with the tests.
3) I think I've nailed down the Half-Close stuff.
Awesome. Do you have a link to a changeset for that? (Did I miss it earlier in the thread?)
4) These are kqueue specific. Need to investigate further.
These are the ones to focus on for http://tm.tl/1918, then :).
twisted.internet.test.test_fdset ReactorFDSetTestsBuilder_KQueueReactor test_lostFileDescriptor ... [ERROR] test_removedFromReactor ... [FAIL]
The first one comes up, since
removeReader()
is called for a FD which is gone, and leads to a socket exception from fd = reader.fileno()
It sounds like you've narrowed this down pretty far! Good work.
5) These are kqueue specific. Need to investigate further.
twisted.conch.test.test_conch
The fail on the kqeue control call .. see below.
Any ideas what could be the reason?
I would suggest fixing the issues you outlined in point 4 first. It's really unfortunate that reactor issues are being caught by conch tests. The reactor tests should be comprehensive enough to catch everything in the reactor.
This may just be a side-effect of the other errors you're tracking down, so see if you can fix those, and then loop back to these. If they disappear, then great. If not, then we should have a test that specifically tests this functionality and doesn't rely on weird and difficult to diagnose errors in conch to find them.
Thanks again for working on this!
-glyph
On Nov 7, 2011, at 10:20 AM, Tobias Oberstein wrote:
Ok, I've prepared
http://twistedmatrix.com/trac/attachment/ticket/1918/kqueue.patch
which passes all tests (FreeBSD 8.3 p3 i386 / Python 2.7.1) but the ones related to open FDs (which are unrelated to the reactor stuff).
This works for me for FreeBSD .. I've tried on MacOSX .. there seem to be PTY issues there. Well.
I've also updated the
http://pypi.python.org/pypi/txkqreactor/0.2
for anyone who wants to try before this gets eventually merged into Twisted.
I've reviewed it, although looking at this email now I realize I missed the 'kqueue.patch' and instead reviewed just the reactor and not the test change. Your patch is totally formatted correctly so please ignore the comments to the contrary on the ticket! And the test change looks like it might fix the failure that I mention in the review, so if you just deal with a few of those nitpicky issues and resubmit with the patch attachment and the 'review' a little closer to each other, I think this may be ready to land in Twisted soon.
If you want to maintain txkqreactor for older users of Twisted, you might want to provide a reactor plugin in your package so that users can do 'twistd --reactor txkq'.
Thanks again, it's great to see progress on this ancient ticket :).
-glyph
participants (6)
-
exarkun＠twistedmatrix.com -
Glyph -
Glyph Lefkowitz
-
Itamar Turner-Trauring -
Kevin Horn -
Tobias Oberstein