On Wed, Jun 17, 2020 at 11:42 AM Emily Bowman <silverbacknet@gmail.com> wrote:
> So most likely there wouldn't be any way to share something like a bytearray or another
> buffer interface-compatible type for some time. That's too bad, I was hoping to have
> shared arrays that I could put a memoryview on in each thread/interpreter and deal with
> locking if I need to,
Earlier versions of PEP 554 did have a "SendChannel.send_buffer()"
method for this but we tabled it in the interest of simplifying. That
said, I expect we'll add something like that separately later.
Right, buffers are different because the receiving interpreter can set up a memoryview that refers to storage allocated by the source interpreter.
So the Python objects aren't shared (avoiding refcounting complications), but the expensive data copying step can still be avoided.
> Packages like NumPy have had their own opaque C types and C-only routines to handle all the big threading outside of Python as a workaround for a long time now.
As a workaround for what? This sounds interesting. :)
For the GIL - lots of NumPy operations are in pure C or FORTRAN and will happily use as many CPUs as you have available.
Cheers,
Nick.