On Thu, 2005-07-07 at 13:48 -0400, Tom Cocagne wrote:

I was wondering if anyone has suggestions for a good way to go about setting up a message-passing communication channel between one or more C threads and a single thread dedicated to running Python/Twisted. I already have functional Python bindings accessing the contents of the messages (basically C structures) but need some way to exchange the messages between the C threads and the Python thread.

Any reason you're using threads? You can run Twisted directly on the glib event loop, so your whole program could run on the same loop. In any case, Switching data from Twisted to C is just like passing data from any Python program to C code: call a C extension function from Python that calls into C code, and for reverse call a Python function from C code. Thread safety is mostly orthogonal, so you can probably use the standard Twisted reactor.callFromThread to have code run in the Twisted thread.

On Thu, Jul 07, 2005 at 01:48:57PM -0400, Tom Cocagne wrote:

All I really need is a means by which to inform Twisted's reactor that a C thread has sent a message. My original plan was to use a pair of pipes (via os.pipe()) to send pointers back and forth but it didn't seem to work on Windows . I'm pretty sure this will work fine for posix systems but it definitely didn't work on my first attempt with Windows (though the problem may have been due to my library configuration).

#define MY_MODULE "foo.bar" #define MY_FUNCTION "handle_message" void send_message_to_python(message_t *message) { PyObject *module_name, *module, *dict, *func, *args, *pymessage; module_name = PyString_FromString(MY_MODULE); module = PyImport_Import(module_name); dict = PyModule_GetDict(module) func = PyDict_GetItemString(MY_FUNCTION); args = PyTuple_New(1) pymessage = MyPythonWrappedMessage_New(message); PyTuple_SetItem(args, 0, pymessage); PyObject_CallObject(func, args); } (with appropriate error checking, of course) This has the effect of doing from foo.bar import handle_message handle_message(m) I'm not at all sure what would happen on Windows. I suppose this would work there too. But you also need to run the reactor, I guess which might be easier or harder depending on the behaviour of the C program. The simplest way is just to start a C thread and do the equivalent of the above for from twisted.internet import reactor reactor.run() -w

Thanks for the quick response guys. A bit more of a description of what I'm working on may help. Basically, I'm working on building a coordination library for use in programs taking part in a distributed simulation. The simulations themselves are written primarily in C++, run on a bunch of different platforms, and are often optomized to use the specific hardware they're running on top of (C-threads locked to specific CPUs, arcane compiler flags, the whole shebang). The Python/Twisted side handles all the high-level coordination stuff while the C++ code focuses on numerics. As the library is supposed to be as generic as possible, I can't make too many assumptions about how threading will be handled in any particular application. So to keep things simple I'm creating a single thread that is dedicated to running the Python interpreter and allowing only that thread to touch the interpreter. I looked into the code for reactor.callFromThread that y'all mentioned and it looks like the implementation for it exactly answers the question I've been trying to ask. On posix systems the reactor's 'waker' is implemented as a pair of pipes and on Win32 systems it's implemented as a pair of sockets. So it appears that my original approach will work except that on Win32 systems I'll just have to use a socket rather than a pipe. Thanks for the help, Tom BTW Itamar, I took a look at the SmartFrog/Coil documentation you sent out yesterday and it looks very promising. Do you have any sort of roadmap/timeline for Coil? On 7/7/05, Itamar Shtull-Trauring wrote:

On Thu, 2005-07-07 at 13:48 -0400, Tom Cocagne wrote:

...

I was wondering if anyone has suggestions for a good way to go about setting up a message-passing communication channel between one or more C threads and a single thread dedicated to running Python/Twisted. I already have functional Python bindings accessing the contents of the messages (basically C structures) but need some way to exchange the messages between the C threads and the Python thread.

Any reason you're using threads? You can run Twisted directly on the glib event loop, so your whole program could run on the same loop.

In any case, Switching data from Twisted to C is just like passing data from any Python program to C code: call a C extension function from Python that calls into C code, and for reverse call a Python function from C code. Thread safety is mostly orthogonal, so you can probably use the standard Twisted reactor.callFromThread to have code run in the Twisted thread.

_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com http://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python