[Python-Dev] Making python C-API thread safe (try 2)

[Shane Hathaway]

Martin v. Löwis wrote:
> Shane Hathaway <shane at zope.com> writes:
> 
> 
>>I wonder whether others would consider such a thing valuable, or even
>>feasible. :-)
> 
> 
> I consider it valuable, but not feasible. Contrary to your analysis, I
> believe the biggest problem with shared memory is pointer/address
> management. E.g. if you have a shared list L, and do
> 
> L.append(1)
> 
> then a reference to the object 1 is put into L[0]. However, the
> pointer is only valid in the process writing the reference, not in
> other processes. Fixing that is nearly impossible, except *perhaps* by
> reimplementing Python from ground up.

There are two ways to solve that:

- Copy the "1" object into shared memory before passing it to 
L.append().  This would be nice and transparent, but also a little DWIM.

- Create special object types just for shared memory.

As it turns out, POSH (see Jeremy's link) takes the second strategy, 
apparently quite successfully!  Anyone who has concerns about the GIL 
should take a long, hard look at POSH.  It just might solve the problem 
for lots of people.

Shane