[Twisted-Python] Callback blocked in event loop
Hello I've got a bit of an issue that someone may recognise and bea ble to set me straight! I have a twisted.spread server which, amongst other things, provides an event pub/sub engine. The subscription manager and event publisher runs in the twisted event loop: an event is posted onto a DeferredQueue and the callback on that Queue iterates through listeners calling eventFired(...). Some event listeners run in-server, but others are seperate processes which register by calling the register function on the server and providing a pb.Referenceable that the event publisher can call eventFired(...) on. Events may be raised in the server, as a twisted.pb call to fireEvent on the server and also come in on a separate thread (from a JMS message queue) and get posted onto the DefferedQueue for events. If events are raised in the server, remote listeners get their pb.Referenceable callback immediately. It works as I would like. However, if a message comes in off the JMS bus, the DeferredQueue callback gets called, it in turns calls the referenceable callback and nothing happens. Only when the server handles *any* other Twisted event do the event callbacks execute and my remote listeners get the message. So, only in the case where teh DeferredQueue.put(...) is called from another thread do I have the problem. I can't see why the blocking. Does this behaviour make sense to anyone!? Many thanks in advance for any help!! Matthew
On Fri, 10 Nov 2006 15:24:51 +0000 (GMT), Matt P <twisted-list@zorinholdings.com> wrote:
Hello
I've got a bit of an issue that someone may recognise and bea ble to set me straight!
[snip]
Events may be raised in the server, as a twisted.pb call to fireEvent on the server and also come in on a separate thread (from a JMS message queue) and get posted onto the DefferedQueue for events.
If events are raised in the server, remote listeners get their pb.Referenceable callback immediately. It works as I would like.
[snip]
So, only in the case where teh DeferredQueue.put(...) is called from another thread do I have the problem. I can't see why the blocking. Does this behaviour make sense to anyone!?
DeferredQueue isn't threadsafe. Instead of calling put() directly from a non-reactor thread, call it with reactor.callFromThread: reactor.callFromThread(q.put, value) Jean-Paul
Oh my - how could I have been so daft? I'll put it down to Friday - thankyou very much Jean-Paul for opening my eyes! Cheers Matthew On Fri, 10 Nov 2006, Jean-Paul Calderone wrote:
On Fri, 10 Nov 2006 15:24:51 +0000 (GMT), Matt P <twisted-list@zorinholdings.com> wrote:
Hello
I've got a bit of an issue that someone may recognise and bea ble to set me straight!
[snip]
Events may be raised in the server, as a twisted.pb call to fireEvent on the server and also come in on a separate thread (from a JMS message queue) and get posted onto the DefferedQueue for events.
If events are raised in the server, remote listeners get their pb.Referenceable callback immediately. It works as I would like.
[snip]
So, only in the case where teh DeferredQueue.put(...) is called from another thread do I have the problem. I can't see why the blocking. Does this behaviour make sense to anyone!?
DeferredQueue isn't threadsafe. Instead of calling put() directly from a non-reactor thread, call it with reactor.callFromThread:
reactor.callFromThread(q.put, value)
Jean-Paul
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
---------------------------------------------- Matthew Pontefract http://www.zorinholdings.com/ ----------------------------------------------
participants (3)
-
Jean-Paul Calderone
-
Matt P
-
MattP