[Twisted-Python] Immediate removal of cfreactor
Hello all, As reported in <http://twistedmatrix.com/trac/ticket/4652>, some problems have arisen with cfreactor. First, recent versions of Pyrex reject the Pyrex source for the support modules. This makes future development on cfreactor difficult (although Glyph's branch does resolve this problem - or at least appears to). Second, the C source files do not build with clang. This seems likely to become more important on future versions of OS X (the only platform where this code matters). Even OS X 10.6 makes clang easily available (according to an Ars Technica article, "Clang is the recommended compiler, and the focus of all of Apple's future efforts." <http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/9>) Finally, when the cfreactor build fails, distutils prematurely exits, preventing the rest of the build from completing. This point makes this a serious problem for Twisted as a whole, rather than an obscure platform quirk that can be ignored without wide-ranging consequences. Apple has announced OS X 10.7 for summer 2011. Presumably at some point prior to this they will freeze on a version of Twisted to package. The best outcome for Twisted can probably be attained by ensuring this is a version of Twisted which does not have the build issues reported in #4652. Rather than leaving this to chance, I propose that we remove cfreactor and all supporting code ourselves, immediately (ie, in 10.2 - hi therve, go faster next time ;). I think the relatively small userbase of cfreactor (I don't know of any projects using it, nor does Google codesearch) and the fact that the version of cfreactor in trunk segfaults partway through running the test suite, combined with the potentially time-sensitive nature of this fix, merits an exception to the normal backwards policy. Rather than removing it, we could fix it, but the fact that no one seems to be using it makes me think that no one with the expertise and interest will step up to so. If we're going to do this, it would be good to do it in the next few days (say, by Tuesday) so as not to unduly further delay the 10.2 release process. Thoughts? Jean-Paul
On Oct 30, 2010, at 8:51 PM, exarkun@twistedmatrix.com wrote:
Rather than removing it, we could fix it, but the fact that no one seems to be using it makes me think that no one with the expertise and interest will step up to so.
I don't actually know enough about the APIs in question to be sure, but my understanding is that the 'cfsupport' pyrex mess was added to paper over some holes in PyObjC, which have (long?) since been filled. At least, I can import all the functions from 'CFNetwork' that it looks like we would need (based on the information here <http://bit.ly/bQWC3J>). If someone wanted to do a real CFNetwork-based reactor, I think this means we wouldn't need to maintain any of our own C code.
On Sun, 2010-10-31 at 00:51 +0000, exarkun@twistedmatrix.com wrote:
Hello all,
As reported in <http://twistedmatrix.com/trac/ticket/4652>, some problems have arisen with cfreactor.
First, recent versions of Pyrex reject the Pyrex source for the support modules. This makes future development on cfreactor difficult (although Glyph's branch does resolve this problem - or at least appears to).
So, the first problem appears to be fixed.
Second, the C source files do not build with clang. This seems likely to become more important on future versions of OS X (the only platform where this code matters). Even OS X 10.6 makes clang easily available (according to an Ars Technica article, "Clang is the recommended compiler, and the focus of all of Apple's future efforts." <http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/9>)
Have you tried seeing if Cython works with Clang? That ought to be a very minor change to our code.
Rather than leaving this to chance, I propose that we remove cfreactor and all supporting code ourselves, immediately (ie, in 10.2 - hi therve, go faster next time ;).
This violates our deprecation policy... so I suggest we only do it if absolutely must, i.e. if Glyph's fixes + Cython don't fix the problem.
On 12:49 pm, itamar@itamarst.org wrote:
On Sun, 2010-10-31 at 00:51 +0000, exarkun@twistedmatrix.com wrote:
Hello all,
As reported in <http://twistedmatrix.com/trac/ticket/4652>, some problems have arisen with cfreactor.
First, recent versions of Pyrex reject the Pyrex source for the support modules. This makes future development on cfreactor difficult (although Glyph's branch does resolve this problem - or at least appears to).
So, the first problem appears to be fixed.
Second, the C source files do not build with clang. This seems likely to become more important on future versions of OS X (the only platform where this code matters). Even OS X 10.6 makes clang easily available (according to an Ars Technica article, "Clang is the recommended compiler, and the focus of all of Apple's future efforts." <http://arstechnica.com/apple/reviews/2009/08/mac-os-x-10-6.ars/9>)
Have you tried seeing if Cython works with Clang? That ought to be a very minor change to our code.
Here are the complete results for all combinations: | trunk | branch | gcc | clang | gcc | clang ------------------------------------------------------- Cython | FTB pyx | FTB pyx | runLoop err | runLoop err ------------------------------------------------------- Pyrex | segfault | FTB c | runLoop err | runLoop err FTB pyx - Failed to build .pyx into .c FTB c - Failed to build .c into .so segfault - crash during test suite run runLoop err - Fails to start reactor because "could not convert runLoop"
Rather than leaving this to chance, I propose that we remove cfreactor and all supporting code ourselves, immediately (ie, in 10.2 - hi therve, go faster next time ;).
This violates our deprecation policy... so I suggest we only do it if absolutely must, i.e. if Glyph's fixes + Cython don't fix the problem.
Quite so. It seems like an okay violation to me, since according to the table above the best possible behavior cfreactor is able to provide is that sometimes it will build and run for a while before segfaulting. Jean-Paul
On Nov 1, 2010, at 9:26 AM, exarkun@twistedmatrix.com wrote:
Quite so. It seems like an okay violation to me, since according to the table above the best possible behavior cfreactor is able to provide is that sometimes it will build and run for a while before segfaulting.
That table is also non-deterministic. I've seen different behavior on different machines, which then changed again when I went back to look again. In no case was I able to get anywhere close to completing a test suite run, though.
participants (3)
-
exarkun＠twistedmatrix.com -
Glyph Lefkowitz -
Itamar Turner-Trauring