[Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

[David Foster]

I was not aware of PyParallel. The PyParellel "parallel thread" 
line-of-execution implementation is pretty interesting. Trent, big kudos 
to you on that effort.

Since you're speaking in the past tense and said "but we're not doing it 
like that", I infer that the notion of a parallel thread was turned down 
for integration into CPython, as that appears to have been the original 
goal.

However I am unable to locate a rationale for why that integration was 
turned down. Was it deemed to be too complex to execute, perhaps in the 
context of providing C extension compatibility? Was there a desire to 
see a similar implementation on Linux as well as Windows? Some other 
reason? Since I presume you were directly involved in the discussions, 
perhaps you have a link to the relevant thread handy?

The last update I see from you RE PyParallel on this list is:
https://mail.python.org/pipermail/python-ideas/2015-September/035725.html

David Foster | Seattle, WA, USA

On 7/9/18 9:17 AM, Trent Nelson wrote:
> On Sun, Jul 08, 2018 at 11:27:08AM -0700, David Foster wrote:
> 
>> I'd like to solicit some feedback on what might be the most
>> efficient way to make forward progress on efficient parallelization
>> in Python inside the same OS process. The most promising areas
>> appear to be:
> 
> You might find PyParallel interesting, at least from a "here's what was
> tried, it worked, but we're not doing it like that" perspective.
> 
>      http://pyparallel.org
>      https://speakerdeck.com/trent/pyparallel-how-we-removed-the-gil-and-exploited-all-cores
> 
> I still think it was a pretty successful proof-of-concept regarding
> removing the GIL without having to actually remove it.  Performance was
> pretty good too, as you can see in those graphs.
> 
>> -- 
>> David Foster | Seattle, WA, USA
> 
> Regards,
> 
>      Trent.
> 
> --
> https://trent.me
>