[Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

Leonardo Santagada santagada at gmail.com
Sun Jul 15 18:49:15 EDT 2018


No one talked about this, but modern cpus with multiple numa nodes are
atrocious to any shared memory (maybe threadripper is better, but
multiple socket xeon is slow) and more and more all cpus will move to
it on single chips, so a share nothing aproach can really make python
a good contender on modern hardware.

On Sun, Jul 15, 2018 at 6:20 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On 11 July 2018 at 00:31, David Foster <davidfstr at gmail.com> wrote:
>> I was not aware of PyParallel. The PyParellel "parallel thread"
>> line-of-execution implementation is pretty interesting. Trent, big kudos to
>> you on that effort.
>>
>> Since you're speaking in the past tense and said "but we're not doing it
>> like that", I infer that the notion of a parallel thread was turned down for
>> integration into CPython, as that appears to have been the original goal.
>>
>> However I am unable to locate a rationale for why that integration was
>> turned down. Was it deemed to be too complex to execute, perhaps in the
>> context of providing C extension compatibility? Was there a desire to see a
>> similar implementation on Linux as well as Windows? Some other reason?
>
> It was never extended beyond Windows, and a Windows-only solution
> doesn't meet the needs of a lot of folks interested in more efficient
> exploitation of multiple local CPU cores.
>
> It's still an interesting design concept though, especially for
> problems that can be deconstructed into a setup phase (read/write main
> thread), and a parallel operation phase (ephemeral worker threads that
> store all persistent state in memory mapped files, or otherwise
> outside the current process).
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 

Leonardo Santagada


More information about the Python-ideas mailing list