[Twisted-Python] Wrapping a Perspective Broker remote object with a blocking API
Hi everyone.
I've done some reading, and it seems that there have been several attempts by new twisted users to get Perspective Broker to function more like Corba or DCOM. Despite having read http://twistedmatrix.com/pipermail/twisted-python/2007-April/thread.html#151... on why this does not work, I have had a go and have got something that seems to work well, at least for my test case.
Server side it is pretty standard. Client side I run the twisted reactor in a separate (non-main) thread, and use threading.Event() objects placed into a 'pass through' callback to emulate blocking until the required deferred has its result.
I have attached a simple server and client, with callback functionality illustrated.
I have not really looked into the thread safety issues yet, but have been looking into reactor.callFromThread and whether it is required here.
If the entire approach is inherently broken for some reason, I'd like to know so I can look at alternatives.
Cheers,
Rasjid.
On Tue, 4 Sep 2007 15:22:37 +1000, Rasjid Wilcox rasjidw@gmail.com wrote:
Hi everyone.
[snip]
Server side it is pretty standard. Client side I run the twisted reactor in a separate (non-main) thread, and use threading.Event() objects placed into a 'pass through' callback to emulate blocking until the required deferred has its result.
I have attached a simple server and client, with callback functionality illustrated.
The example code doesn't do anything with Events, and I'm not sure I understand what you're describing above, so I don't know if that part of your code is correct or not.
I have not really looked into the thread safety issues yet, but have been looking into reactor.callFromThread and whether it is required here.
Yes, it is. As your code is now, it calls APIs which are not threadsafe from a non-reactor thread. This isn't allowed and will definitely result in incorrect behavior sometimes.
If the entire approach is inherently broken for some reason, I'd like to know so I can look at alternatives.
I'm not sure why you're trying to do this at all, instead of just using Twisted in a single-threaded manner.
Jean-Paul
I'm not sure why you're trying to do this at all, instead of just using Twisted in a single-threaded manner.
This is a usage case that we run into regularly. We have a lot of PB based servers, but sometimes people want to use a client to them from an interactive Python or IPython prompt. At that point you have to run the reactor in a different thread. But, I have never seen a method of putting a blocking API on top of such a client that was _really_ thread safe. I have seen a few approaches that seem to work, but that are clearly non-thread safe. I would love to see a robust solution for this problem though - one that could be fully trusted.
Brian
Jean-Paul
Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Tue, 4 Sep 2007 09:38:12 -0600, Brian Granger ellisonbg.net@gmail.com wrote:
I'm not sure why you're trying to do this at all, instead of just using Twisted in a single-threaded manner.
This is a usage case that we run into regularly. We have a lot of PB based servers, but sometimes people want to use a client to them from an interactive Python or IPython prompt. At that point you have to run the reactor in a different thread.
Not true, in fact. See twisted/conch/stdio.py, for example. I don't know what the Twisted/IPython integration code ended up looking like (or if it was ever taken to a satisfactory "completion" point) but I don't think it is necessary to have multiple threads for that use case, either.
But, I have never seen a method of putting a blocking API on top of such a client that was _really_ thread safe. I have seen a few approaches that seem to work, but that are clearly non-thread safe. I would love to see a robust solution for this problem though - one that could be fully trusted.
Why isn't a solution based on Twisted's thread-safe event-posting API (reactor.callFromThread) and the thread-safe event-posting API of whatever other thing is being integrated with "really" thread-safe? Sure, you can write programs which will deadlock as a result of application-level bugs where one thread is blocked on a request and the other thread needs some information from it before the request can be satisfied, but this is inherent to wanting to drive an asynchronous API synchronously. Aside from it, I don't think there are any problems with what I described above.
Jean-Paul
This is a usage case that we run into regularly. We have a lot of PB based servers, but sometimes people want to use a client to them from an interactive Python or IPython prompt. At that point you have to run the reactor in a different thread.
Not true, in fact. See twisted/conch/stdio.py, for example. I don't know what the Twisted/IPython integration code ended up looking like (or if it was ever taken to a satisfactory "completion" point) but I don't think it is necessary to have multiple threads for that use case, either.
I have, but this doesn't solve the problem as I recall. First, these users want to be able to use PB clients from vanilla python interpreters and IPython out of the box - not from within some other process doing the tricks that are in conch/stdio.py. Am I not correct that threads are needed in this case? Second, what this module does is actually make the terminal asynchronous, which is exactly the opposite of what these users want. They *want* a synchronous terminal and a blocking PB clients. I hope I am not coming across as angry about this - I'm not. To me the lesson is that as long as everything is really asynchronous anything is possible within the constraints of this asynchronous universe. It's the multiple universe things that makes life complicated (asyn + syn).
But, I have never seen a method of putting a blocking API on top of such a client that was _really_ thread safe. I have seen a few approaches that seem to work, but that are clearly non-thread safe. I would love to see a robust solution for this problem though - one that could be fully trusted.
Why isn't a solution based on Twisted's thread-safe event-posting API (reactor.callFromThread) and the thread-safe event-posting API of whatever other thing is being integrated with "really" thread-safe? Sure, you can write programs which will deadlock as a result of application-level bugs where one thread is blocked on a request and the other thread needs some information from it before the request can be satisfied, but this is inherent to wanting to drive an asynchronous API synchronously. Aside from it, I don't think there are any problems with what I described above.
It probably is, but I have never seen an implementation that actually does this. Do you know of one? The ones that I have seen - even those which use callFromThread have not filled me with thoughts of robustness, stability and thread safety. I should look at this again though.
Also, I should mention that in IPython this is not a show stopper for us, but it does mean that we currently don't use PB in all of the places we might otherwise.
Brian
Jean-Paul
Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
I take most of this back. I just saw that in twisted.internet.threads there is now a
blockingCallFromThread
function that does this exactly. Last time I looked at the ticket that led to this, there were still some things being worked out. This look very nice though!
Brian
On 9/4/07, Brian Granger ellisonbg.net@gmail.com wrote:
This is a usage case that we run into regularly. We have a lot of PB based servers, but sometimes people want to use a client to them from an interactive Python or IPython prompt. At that point you have to run the reactor in a different thread.
Not true, in fact. See twisted/conch/stdio.py, for example. I don't know what the Twisted/IPython integration code ended up looking like (or if it was ever taken to a satisfactory "completion" point) but I don't think it is necessary to have multiple threads for that use case, either.
I have, but this doesn't solve the problem as I recall. First, these users want to be able to use PB clients from vanilla python interpreters and IPython out of the box - not from within some other process doing the tricks that are in conch/stdio.py. Am I not correct that threads are needed in this case? Second, what this module does is actually make the terminal asynchronous, which is exactly the opposite of what these users want. They *want* a synchronous terminal and a blocking PB clients. I hope I am not coming across as angry about this - I'm not. To me the lesson is that as long as everything is really asynchronous anything is possible within the constraints of this asynchronous universe. It's the multiple universe things that makes life complicated (asyn + syn).
But, I have never seen a method of putting a blocking API on top of such a client that was _really_ thread safe. I have seen a few approaches that seem to work, but that are clearly non-thread safe. I would love to see a robust solution for this problem though - one that could be fully trusted.
Why isn't a solution based on Twisted's thread-safe event-posting API (reactor.callFromThread) and the thread-safe event-posting API of whatever other thing is being integrated with "really" thread-safe? Sure, you can write programs which will deadlock as a result of application-level bugs where one thread is blocked on a request and the other thread needs some information from it before the request can be satisfied, but this is inherent to wanting to drive an asynchronous API synchronously. Aside from it, I don't think there are any problems with what I described above.
It probably is, but I have never seen an implementation that actually does this. Do you know of one? The ones that I have seen - even those which use callFromThread have not filled me with thoughts of robustness, stability and thread safety. I should look at this again though.
Also, I should mention that in IPython this is not a show stopper for us, but it does mean that we currently don't use PB in all of the places we might otherwise.
Brian
Jean-Paul
Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
I take most of this back. I just saw that in twisted.internet.threads there is now a
blockingCallFromThread
function that does this exactly. Last time I looked at the ticket that led to this, there were still some things being worked out. This look very nice though!
Brian
On 9/4/07, Brian Granger ellisonbg.net@gmail.com wrote:
This is a usage case that we run into regularly. We have a lot of PB based servers, but sometimes people want to use a client to them from an interactive Python or IPython prompt. At that point you have to run the reactor in a different thread.
Not true, in fact. See twisted/conch/stdio.py, for example. I don't know what the Twisted/IPython integration code ended up looking like (or if it was ever taken to a satisfactory "completion" point) but I don't think it is necessary to have multiple threads for that use case, either.
I have, but this doesn't solve the problem as I recall. First, these users want to be able to use PB clients from vanilla python interpreters and IPython out of the box - not from within some other process doing the tricks that are in conch/stdio.py. Am I not correct that threads are needed in this case? Second, what this module does is actually make the terminal asynchronous, which is exactly the opposite of what these users want. They *want* a synchronous terminal and a blocking PB clients. I hope I am not coming across as angry about this - I'm not. To me the lesson is that as long as everything is really asynchronous anything is possible within the constraints of this asynchronous universe. It's the multiple universe things that makes life complicated (asyn + syn).
But, I have never seen a method of putting a blocking API on top of such a client that was _really_ thread safe. I have seen a few approaches that seem to work, but that are clearly non-thread safe. I would love to see a robust solution for this problem though - one that could be fully trusted.
Why isn't a solution based on Twisted's thread-safe event-posting API (reactor.callFromThread) and the thread-safe event-posting API of whatever other thing is being integrated with "really" thread-safe? Sure, you can write programs which will deadlock as a result of application-level bugs where one thread is blocked on a request and the other thread needs some information from it before the request can be satisfied, but this is inherent to wanting to drive an asynchronous API synchronously. Aside from it, I don't think there are any problems with what I described above.
It probably is, but I have never seen an implementation that actually does this. Do you know of one? The ones that I have seen - even those which use callFromThread have not filled me with thoughts of robustness, stability and thread safety. I should look at this again though.
Also, I should mention that in IPython this is not a show stopper for us, but it does mean that we currently don't use PB in all of the places we might otherwise.
Brian
Jean-Paul
Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
On Tue, 4 Sep 2007 12:45:17 -0600, Brian Granger ellisonbg.net@gmail.com wrote:
This is a usage case that we run into regularly. We have a lot of PB based servers, but sometimes people want to use a client to them from an interactive Python or IPython prompt. At that point you have to run the reactor in a different thread.
Not true, in fact. See twisted/conch/stdio.py, for example. I don't know what the Twisted/IPython integration code ended up looking like (or if it was ever taken to a satisfactory "completion" point) but I don't think it is necessary to have multiple threads for that use case, either.
I have, but this doesn't solve the problem as I recall.
There are definitely some problems it doesn't solve. Maybe I'm not getting the problem you have in mind.
First, these users want to be able to use PB clients from vanilla python interpreters and IPython out of the box - not from within some other process doing the tricks that are in conch/stdio.py.
stdio.py doesn't do anything with multiple processes (as I think you see, based on what you said a couple sentences later, but just to be clear).
Another point to clarify - there is a difference between having a blocking user interface and a blocking API. In fact, they're different things entirely - they don't relate to each other in any significant way.
Keeping this in mind, there's no reason either the standard CPython REPL or IPython couldn't provide a blocking user interface to an asynchronous API, without resorting to multiple processes or threads. The implementation is asynchronous, the user interface blocks.
Of course, actually implementing this for the CPython REPL poses some challenges. However, an API does exist which would allow it to poll Twisted at regular intervals in order to allow Twisted to take care of any event processing it needed to take care of. Whether this would yield a high- quality implementation or not, I'm not sure, but something should be possible. The reason I wrote stdio.py instead of doing this is that I'd rather have a REPL implemented in Python without having to jump through libreadline hoops. Plus I didn't know about the CPython polling feature at the time. ;)
IPython could do this even more easily, since I think some IPython developers are actively interested in the feature, and there might be a chance of there being some changes made to IPython in order to facilitate this.
I hope I have understood properly the use-case you have in mind and that the above makes some sense.
Am I not correct that threads are needed in this case? Second, what this module does is actually make the terminal asynchronous, which is exactly the opposite of what these users want.
stdio.py keeps everything asynchronous and throws Deferreds right in the user's face. But it does this because I'm a crazy guy and I love Deferreds. It's equally possible to implement something like stdio.py, without using threads or multiple processes, where the user experiences the interface blocking until a result is available each time he would have otherwise seen a Deferred.
They *want* a synchronous terminal and a blocking PB clients. I hope I am not coming across as angry about this - I'm not.
Not at all. :)
To me the lesson is that as long as everything is really asynchronous anything is possible within the constraints of this asynchronous universe. It's the multiple universe things that makes life complicated (asyn + syn).
Does the notion of dividing things up so that user interface concerns are dealt with separately from API concerns clear this up at all? Do you still think there's something here which can't be solved without threads?
(To be sure, there is something - if you have a blocking API and you want to use it inside an event loop, then you may need to resort to threads, but I don't *think* that's the case you're interested in here - correct me if I'm wrong.)
But, I have never seen a method of putting a blocking API on top of such a client that was _really_ thread safe. I have seen a few approaches that seem to work, but that are clearly non-thread safe. I would love to see a robust solution for this problem though - one that could be fully trusted.
Why isn't a solution based on Twisted's thread-safe event-posting API (reactor.callFromThread) and the thread-safe event-posting API of whatever other thing is being integrated with "really" thread-safe? Sure, you can write programs which will deadlock as a result of application-level bugs where one thread is blocked on a request and the other thread needs some information from it before the request can be satisfied, but this is inherent to wanting to drive an asynchronous API synchronously. Aside from it, I don't think there are any problems with what I described above.
It probably is, but I have never seen an implementation that actually does this. Do you know of one? The ones that I have seen - even those which use callFromThread have not filled me with thoughts of robustness, stability and thread safety. I should look at this again though.
Ah, well, I don't have any that I can point too and claim are robust and bug free (or even bug lite). But hey, the *idea* is simple enough. >:)
Also, I should mention that in IPython this is not a show stopper for us, but it does mean that we currently don't use PB in all of the places we might otherwise.
Is this because you want to have blocking APIs in order to implement features in IPython or is it because you want to present an interface to users which behaves in a blocking manner? (Sorry if this is beginning to sound repetitive)
Jean-Paul
On 9/4/07, Jean-Paul Calderone exarkun@divmod.com wrote:
The example code doesn't do anything with Events, and I'm not sure I understand what you're describing above, so I don't know if that part of your code is correct or not.
From simple-client.py
def wait_factory(): "Returns a callback to apply to a deffered, and event to wait on" evt = threading.Event() def set_event(result): evt.set() return result return set_event, evt
I have not really looked into the thread safety issues yet, but have been looking into reactor.callFromThread and whether it is required here.
Yes, it is. As your code is now, it calls APIs which are not threadsafe from a non-reactor thread. This isn't allowed and will definitely result in incorrect behavior sometimes.
I figured that was the case. I was planning to fix this after getting some feedback. Which has been worthwhile, since I now know about twisted.internet.blockingCallFromThread which would appear to do what I want.
If the entire approach is inherently broken for some reason, I'd like to know so I can look at alternatives.
I'm not sure why you're trying to do this at all, instead of just using Twisted in a single-threaded manner.
Because the ultimate 'user' of the remote object will be a Visual Basic program.
My vision is as follows: The Visual Basic program will talk to a python com server. The python com server will run a PB client that will talk to a PB server. The PB server will start a windows com component. Everything gets relayed in a fairly transparent manner, and as far as the VB program is concerned, it is just talking to a local com component.
Standard DCOM does not really do what I want, since it assumes a fairly fixed Client-Server model, whereas I want a peer-to-peer setup where each node could be a client and a server. PB would allow me to do exactly that.
It might be possible to create a new twisted reactor that integrates into the python com server event loop. That would presumably be the correct way to deal with this. But that would seem like far more work that I have time to devote to this problem at this time.
So my question really is: What are the problems / brawbacks with running the twisted reactor in a non-main thread, provided all calls to the reactor are done through twisted.internet.blockingCallFromThread?
Cheers,
Rasjid.
participants (3)
-
Brian Granger -
Jean-Paul Calderone -
Rasjid Wilcox