[Python-ideas] Parallel processing with Python

[Sturla Molden]

> On Thu, Feb 19, 2009 at 2:53 AM, Sturla Molden <sturla at molden.no> wrote:

> Only if you have an approach to GC that does not require locking. The
> current reference counting macros are not thread-safe so every INCREF
> and DECREF would have to be somehow protected by a lock or turned into
> an atomic operation. Recall that many frequently used objects, like
> None, small integers, and interned strings for common identifiers, are
> shared and constantly INCREFed and DECREFed.

Thanks for the info.

I think this would be just a minor inconvinience. Sending a message in the
form of PyObject *x from A to B could perhaps be solved like this:

Interpreter A:
   Increfs immutable pyobj x
   aquires the GIL of interpreter B
   messages pyobject x to interpreter B
   releases the GIL of interpreter B

Interpreter B:
   Creates a proxy object p for reading attributes of x
   Increfs & decrefs p
   (refcounts for x or its attributes are not touched by p)
   when p is collected:
      aquires the GIL of interpreter A
      decrefs x
      releases the GIL of interpreter A

Synchronization of reference counting is obviously needed (and the GIL is
great for that). But serialization of the whole object would be avoided.

This would depend on immutability of the message object.


S.M.