[Twisted-Python] Re: [Twisted-commits] Somehow the waker still messes up cfreactor, never installing waker for now
On Tue, 11 Nov 2003 21:43:23 -0700 etrepum CVS <etrepum@wolfwood.twistedmatrix.com> wrote:
Modified files: Twisted/twisted/internet/cfreactor.py 1.3 1.4
Log message: Somehow the waker still messes up cfreactor, never installing waker for now
That means threaded programs with cfreactor will be really slow. I think the issue is that the unix waker uses python sockets. You probably want to write your own waker using whatever APIs you are using. While we're at it - warner, could we have the mac os x reactor also run tests with cfreactor? -- Itamar Shtull-Trauring http://itamarst.org/ Available for Python & Twisted consulting
On Nov 12, 2003, at 12:22 AM, Itamar Shtull-Trauring wrote:
On Tue, 11 Nov 2003 21:43:23 -0700 etrepum CVS <etrepum@wolfwood.twistedmatrix.com> wrote:
Modified files: Twisted/twisted/internet/cfreactor.py 1.3 1.4
Log message: Somehow the waker still messes up cfreactor, never installing waker for now
That means threaded programs with cfreactor will be really slow.
I think the issue is that the unix waker uses python sockets. You probably want to write your own waker using whatever APIs you are using.
While we're at it - warner, could we have the mac os x reactor also run tests with cfreactor?
I don't know what the deal is, exactly, because it uses python sockets internally and that works perfectly fine. I believe that something about the waker causes the run loop to think that it's always awake. I didn't feel like looking too far into it. That said, Apple's implementation of CFRunLoop does everything that the reactor does (and then some), including thread waking and other thread communication facilities.. so when I have more time I'll look into the cause and the appropriate solution. IIRC the only part of Twisted that uses threads on its own is the DB stuff, anything else would probably be implemented using more native means (nobody uses cfreactor for cross-platform applications yet), so I don't see an immediate problem with ignoring the waker. I'd rather punish threaded (from what Twisted sees) applications than make all applications really really slow and resource hungry. -bob
Bob Ippolito wrote:
I don't know what the deal is, exactly, because it uses python sockets internally and that works perfectly fine. I believe that something about the waker causes the run loop to think that it's always awake. I didn't feel like looking too far into it. That said, Apple's implementation of CFRunLoop does everything that the reactor does (and then some), including thread waking and other thread communication facilities.. so when I have more time I'll look into the cause and the appropriate solution. IIRC the only part of Twisted that uses threads on its own is the DB stuff, anything else would probably be implemented using more native means (nobody uses cfreactor for cross-platform applications yet), so I don't see an immediate problem with ignoring the waker. I'd rather punish threaded (from what Twisted sees) applications than make all applications really really slow and resource hungry.
If there are better thread communication facilities in the CFRunLoop, then make your reactor's call[From/In]Thread use them. There's nothing intrinsically useful about the waker; it's an implementation detail of callFromThread which could be better done in another way. However, "really slow" doesn't quite describe the problem. A server which sends an event to a thread without a waker and then receives a callFromThread may never wake up, if there isn't more network activity happening.
On Wed, 12 Nov 2003 01:56:25 -0500 Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
However, "really slow" doesn't quite describe the problem. A server which sends an event to a thread without a waker and then receives a callFromThread may never wake up, if there isn't more network activity
happening.
Yep, a reactor without a waker is essentially broken. It'll also break the SIGCHILD handler in CVS. Please do try to find a solution before our next release. -- Itamar Shtull-Trauring http://itamarst.org/ Available for Python & Twisted consulting
On Nov 12, 2003, at 10:47 AM, Itamar Shtull-Trauring wrote:
On Wed, 12 Nov 2003 01:56:25 -0500 Glyph Lefkowitz <glyph@twistedmatrix.com> wrote:
However, "really slow" doesn't quite describe the problem. A server which sends an event to a thread without a waker and then receives a callFromThread may never wake up, if there isn't more network activity
happening.
Yep, a reactor without a waker is essentially broken. It'll also break the SIGCHILD handler in CVS. Please do try to find a solution before our next release.
Signals need a whole different approach because signals do not wake up CFRunLoop, so when you install handlers it wakes up once a second anyway. The correct solution is to use a mach port waker for signals, I have not implemented this yet because there are no mach port facilities available from Python yet (I looked into it, but doing mach ports use a lot of C macros, so it would be hard). When is next release planned? -bob
On Wed, 12 Nov 2003 11:24:06 -0500 Bob Ippolito <bob@redivi.com> wrote:
When is next release planned?
Doesn't seem like anyone is in a real rush to have one. -- Itamar Shtull-Trauring http://itamarst.org/ Available for Python & Twisted consulting
participants (3)
-
Bob Ippolito -
Glyph Lefkowitz -
Itamar Shtull-Trauring