The C++ library is completely independent of Python and Twisted but it can be made to work with them. One of the initial uses of it, in fact, was to run a Python interpreter (using Twisted) in a dedicated C++ thread and use it to direct the actions of all the other threads in the process. The way I did it is this: 1. Spawn a C++ thread to run the Python interpreter inside of. All use of the Python C-API must be done from within this thread if you want to avoid dealing with the Global Interpreter Lock (remember Python itself is a single-threaded application). Other C++ threads needing to call a Python C-API function can do so using a DeferredCommand object that wraps the call they want to make. Hand this object to the Python thread for execution and use the Deferred mechanism to get the answer back to the original thread..... Basically, you turn the interpreter into one big Active Object as per the standard design pattern. 2. Use the Python C-API to import a Python module you wrote that will initialize Twisted and create a pair of TCP sockets that are connected to each other. Then use the C-API to get the OS-level file descriptor to one of the sockets. The purpose of this is to provide a mechanism for other C++ threads to inform the Twisted reactor that there's something in the C++ world it needs to do. When one of those DeferredCommands in step one is given to the Python thread, the last thing the delivering thread needs to do is call a function that writes a single byte to the TCP socket. This will cause the Twisted reactor in the Python thread to wake up from it's 'select' call and dispatch a handler. This handler can then execute a C++ function you registered with the C-API. Once back in the C++ world and in the context of the Python thread, you can dispatch any of the DeferredCommands sitting in the thread's queue. I don't currently have a mechanism for tying a Twisted Deferred directly to one of the C++ deferreds since I haven't yet found a use-case where it makes sense to do so. It's probably possible though. As for using the C++ deferreds with boost's asio... I haven't used asio yet but a quick read through the API makes it looks like an almost perfect match for these deferreds. In fact, I'd like to eventually put this code up for consideration by the boost community for eventually including something like it in boost. I *highly* doubt they'd want to use the code I've written but it might serve as a good proof-of-concept at least. Tom On Fri, Jan 30, 2009 at 12:09 AM, V S P <toreason@fastmail.fm> wrote:
Hi, this is very interesting ( am I currently testing out C++ web server (using boost asio) called pion to see if can suit my needs (it is a multi-threaded async IO generic server and boost asio uses the best async APIs for each OS (including kques for freebsd for example).
I looked at the example threads.cpp -- but could not figure out -- how would it integrate with Twisted ?
thanks
On Thu, 29 Jan 2009 12:21:36 -0800, "Tom Cocagne" <tom.cocagne@gmail.com> said:
The question of whether anyone had developed a C++ implementation of Twisted's Deferreds has cropped up on this list several times over the last few years. To date, I've unfortunately been forced to sit on an implementation I developed several years ago for the Department of Defense. Due to some rather draconian IP & security policies, it's taken an exceptionally long time to get the code released. I'm happy to announce though that I finally managed to obtain release authorization for it and that the code now has a new home on Sourceforge.
As C++ has differing language features, the API of the C++ deferreds isn't identical to those of Twisted's but most of the key concepts mapped over pretty well. Additionally, this implementation has a few design extensions to facilitate the development multi-threaded C++ applications. The multi-threaded aspect grew almost accidentally out of the original single-threaded design but, since then, it has proven to be extraordinarily useful for simplifying several multi-threaded applications I've worked on.
The implementation of the library is, however, rather complex. In large part, this is due to the effort required to work around the inherent inflexibility of the C++ type system. The Function, Bind, and Meta-Programming libraries found in Boost have gone a long way towards simplify the implementation though (my original Loki-based attempt was almost incomprehensible even to me ;-). The library is fully documented though so as long as you're reasonably comfortable with C++ templates, it should be understandable.
So far, the library has only been exposed to a handful of individuals for review. I'd be interested to hear the comments anyone has about the design and/or implementation approach. The code is available at:
http://deferred.svn.sourceforge.net/viewvc/deferred/
Cheers,
Tom
-- V S P toreason@fastmail.fm
-- http://www.fastmail.fm - Accessible with your email software or over the web
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python