Components for a client/server architecture

[Josiah Carlson]

Samuel wrote:
> On Mon, 21 May 2007 12:06:50 +0200, Diez B. Roggisch wrote:
> 
>> I'm not sure which configuration you want to change how often. But I'm
>> not convinced that the python threading limitations really do make a
>> difference here. Do you really benefit from multi-core capabilities in
>> this scenario?
> 
> The threading issues are not bound to multi cpu systems. The problem is 
> that some of Python's blocking functions require holding the global lock.
> 
> "Not all built-in functions that may block waiting for I/O allow other 
> threads to run."
> "It is not possible to interrupt the acquire() method on a lock"
> http://docs.python.org/lib/module-thread.html

You really should read the source.

static PyObject *
lock_PyThread_acquire_lock(lockobject *self, PyObject *args)
{
	int i = 1;

	if (!PyArg_ParseTuple(args, "|i:acquire", &i))
		return NULL;

	Py_BEGIN_ALLOW_THREADS
	i = PyThread_acquire_lock(self->lock_lock, i);
	Py_END_ALLOW_THREADS

	return PyBool_FromLong((long)i);
}

That snippet of code shows that acquiring a lock does release the GIL. 
Whether or not you are able to interrupt the underlying operating system 
lock acquisition is platform dependent (on *nix, you can signal anything 
to die).  Now, whether or not some other code does or does not release 
the GIL depends on its implementation.  However, having used threads on 
Windows and Linux, with files, sockets, databases, etc., I haven't ever 
experienced a case where the threads did something that wasn't 
reasonable except in once case*.


> I also found that IronPython does not have a global lock, so far it seems 
> well suited for solving the problems I am trying to avoid. I am still 
> looking for a good comparison between IronPython, Python, and Jython.

 From what I've been reading over the last couple years, IronPython is 
relatively competitive with Python, being faster or slower depending on 
the things you are doing.  Jython is syntactically limited to Python 2.2 
at present, so if you want decorators, descriptors, etc., you are out of 
luck.


>> Sounds like CORBA to me. CORBA has a very mature and good implementation
>> for Python called OmniORB, and interoperability with other orbs (the
>> ones available for e.g. Java) is very good - as CORBA as standard is
>> mature.
> 
> I have worked with Bonobo (an implementation of CORBA) before, though not 
> on the network - it is fairly complex. But I did not think of using it 
> for this purpose, it might actually make sense. I'll have to look into 
> the transport protocol more.

You should also consider XML-RPC.  Setting up and using XML-RPC in 
Python is quite easy (see the recipes in the Python cookbook), both as a 
server and client.  If you are thinking about running the server in 
Jython or IronPython anyways, you can always use the standard library 
XML-RPC libraries from Python, aside from the sockets stuff, it's all in 
Python.


  - Josiah

  * That one case was with mixing a 3rd party library that seemed to 
have an incomplete Python wrapping that caused values from two thread 
stacks to be exchanged.  The only way I was able to tickle it was with 
older versions of wxPython + wx.FileConfig + Scintilla + Python's 
compiler module.