[Python-ideas] The future of Python parallelism. The GIL. Subinterpreters. Actors.

[Nathaniel Smith]

On Wed, Jul 18, 2018 at 11:49 AM, Stephan Houben <stephanh42 at gmail.com> wrote:
> Basically, what I am suggesting is a direct translation of Javascript's
> Web Worker API
> (https://developer.mozilla.org/en-US/docs/Web/API/Web_Workers_API)
> to Python.
>
> The Web Worker API is generally considered a "share-nothing" approach,
> although
> as we will see some state can be shared.
>
> The basic principle is that any object lives in a single Worker (Worker =
> subinterpreter).
> If a message is send from Worker A to Worker B, the message is not shared,
> rather the so-called "structured clone" algorithm is used to create
> recursively a NEW message
> object in Worker B. This is roughly equivalent to pickling in A and then
> unpickling in B,
>
> Of course, this may become a bottleneck if large amounts of data need to be
> communicated.
> Therefore, there is a special object type designed to provide a view upon a
> piece
> of shared memory:  SharedArrayBuffer. Notable, this only provides a view
> upon
> raw "C"-style data (ints or floats or whatever), not on Javascript objects.

Note that this everything you said here also exactly describes the
programming model for the existing 'multiprocessing' module:
"structured clone" is equivalent to how multiprocessing uses pickle to
transfer arbitrary objects, or you can use multiprocessing.Array to
get a shared view on raw "C"-style data.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org