[Python-ideas] solving multi-core Python

[Eric Snow]

On Sun, Jun 21, 2015 at 7:47 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> It occurred to me in the context of another conversation that you (or
> someone else!) may be able to prototype some of the public API ideas
> for this using Jython and Vert.x: http://vertx.io/

I'll take a look.

>
> That idea and some of the initial feedback in this thread also made me
> realise that it is going to be essential to keep in mind that there
> are key goals at two different layers here:
>
> * design a compelling implementation independent public API for CSP
> style programming in Python
> * use subinterpreters to implement that API efficiently in CPython
>
> There's a feedback loop between those two goals where limitations on
> what's feasible in CPython may constrain the design of the public API,
> and the design of the API may drive enhancements to the existing
> subinterpreter capability, but we shouldn't lose sight of the fact
> that they're *separate* goals.

Yep.  I've looked at it that way from the beginning.  When I get to
the point of writing an actual PEP, I'm thinking it will actually be
multiple PEPs covering the different pieces.

I've also been considering how to implement that high-level API in
terms of a low-level API (threading vs. _thread) and it it make sense
to focus less on subinterpreters in that context.  At this point it
makes sense to me to expose subinterpreters in Python, so for now I
was planning on that for the low-level API.

-eric