[Python-Dev] Python Interpreter Thread Safety?

[Evan Jones]

On Jan 28, 2005, at 20:27, Tim Peters wrote:
> The hacks in PyObject_Free (== PyMem_Free)
> are there solely so that question can be answered correctly in the
> absence of holding the GIL.   "That question" == "does pymalloc
> control the pointer passed to me, or does the system malloc?".

<Really loud sound of me smacking my hand on my forehead>

Ah! *Now* I get it. And yes, it will be possible to still support this 
in my patched version of the allocator. It just means that I have to 
leak the "arenas" array just like it did before, and then do some hard 
thinking about memory models and consistency to decide if the "arenas" 
pointer needs to be volatile.

> When pymalloc was new, we went to insane lengths to
> avoid breaking that stuff, but enough is enough.

So you don't think we need to bother supporting that any more?

> Back to the present:
>> Wouldn't it be up to the programmer to ensure that accesses to shared
>> objects, like containers, are serialized? For example, with Java's
>> collections, there are both synchronized and unsynchronized versions.
> Enormous mounds of existing threaded Python code freely manipulates
> lists and dicts without explicit locking now.  We can't break that --
> and wouldn't want to.  Writing threaded code is especially easy (a
> relative stmt, not absolute) in Python because of it.

Right, because currently Python switches threads on a granularity of 
opcodes, which gives you this serialization with the cost of never 
having parallel execution.

Evan Jones