Thread safety of the python API

[Donn Cave]

Quoth sludin at ludin.org (Stephen Ludin):
| I am developing an application that I may want to make scriptable with
| python.  If I do so I'd like to use the convenience of the Python API
| in the rest of my application for containers and user defined method
| dispatch.  I am concerned though about using the API in a
| multit-hreaded program.  For example, a browse of intobject.c shows:
|
| static PyIntObject *free_list = NULL;
| ...
| PyObject *
| PyInt_FromLong(long ival)
| {
| ...
| 	if (free_list == NULL) {
| 		if ((free_list = fill_free_list()) == NULL)
| 			return NULL;
| 	}
| 	/* PyObject_New is inlined */
| 	v = free_list;
| 	free_list = (PyIntObject *)v->ob_type;
| ...
| }
|
| The use of the linked list 'free_list' could result in a race
| condition in MT code.  There are a number of similar cases in the
| objects defined in the Objects directory.
|
| I have not found much information on this, so I am wondering if I am
| missing something.  If all of my use was limited to embedded
| interpreters, the global interpreter lock would take care of
| synchronization, but I want to use it in the rest of my program as
| well.

Can you explain what that means, ``but I want to use it in the rest
of my program''?  Do you want to call PyInt_FromLong from outside
the flow of control of the interpreter?  That sounds like something
one would not want to do, as I see Tim Peters has pointed out already.
But nor does it sound like anything one would want to do, if you
see what I mean.

In my multithreaded applications, most of the action in any thread
is outside of the interpreter, in the sense that a C++ library
function is executing that was not specifically designed to be
called from python (and in fact the threads were even started
from that library, not by threadmodule.so.)  While that's going
on, the interpreter lock passes to some other thread, but that's
fine because the library function is going to leave the python
internals alone.  When control returns to where we start doing
python internals, it has to wait for the lock.

	Donn Cave, donn at drizzle.com