[Python-ideas] easy thread-safety [was: fork]

[Chris Angelico]

On Wed, Aug 19, 2015 at 3:17 AM, Sven R. Kunze <srkunze at mail.de> wrote:
> On 18.08.2015 18:55, Chris Angelico wrote:
>>
>> The notion of "a completely separate heap for each thread" is talking
>> about part B - you need a completely separate pile of objects.
>
>
> Right. However, only if needed. As long as, threads only reads from a common
> variable, there is no need to interfere.

Sure, but as soon as you change something, you have to
thread-local-ify it. So I suppose what you would have is three
separate pools of objects:

1) Thread-specific objects, which are referenced only from one thread.
These can be read and written easily and cheaply.
2) Global objects which have never been changed in any way since
threading began. These can be read easily, but if written, must be
transformed into...
3) Thread-local objects, which exist for all threads, but are
different. The id() of such an object depends on which thread is
asking.

Conceptually, all three types behave the same way - changes are
visible only within the thread that made them. But the implementation
could have these magic "instanced" objects for only those ones which
have actually been changed, and save a whole lot of memory for the
others.

> I agree, behavior-wise, processes behave almost as desired (relevant data is
> copied over and there are no shared variables).
>
> However, regarding the cpu/memore/communication footprint for a new process
> (using spawn) is enormous compared to a thread. So, threading still have its
> merits (IMHO).

So really, you're asking for process semantics, with some
optimizations to take advantage of the fact that most of the processes
are going to be just reading and not writing. That may well be
possible, using something like the above three-way-split, but I'd want
the opinion of someone who's actually implemented something like this
- from me, it's just "hey, wouldn't this be cool".

>> There's no way to make module globals thread-local without making the
>> module object itself thread-local, and if you do that, you probably
>> need to make every object it references thread-local too, etc, etc,
>> etc.
>
> Something wrong with that? Shouldn't matter as long as there is only a
> single thread.

As long as there's only a single thread, there's no difference between
process-wide and thread-local. Once you start a second thread,
something needs to know what objects belong where. That's all.

ChrisA