On Jun 20, 2015 9:38 PM, "Steven D'Aprano" <steve@pearwood.info> wrote:
>
> On Sat, Jun 20, 2015 at 03:42:33PM -0600, Eric Snow wrote:
>
> > * only allow passing plain functions to Task() and
> > Subinterpreter.run() (exclude closures, other callables)
>
> That doesn't sound very Pythonic to me. That's going to limit the
> usefulness of these subinterpreters.

It certainly would limit their usefulness.  It's a tradeoff to make the project tractable.  I'm certainly not opposed to dropping such restrictions, now or as a follow-up project.  Also keep in mind that the restriction is only something I'm considering.  It's too early to settle on many of these details.

>
>
> > * object ownership model
> >   + read-only in all but 1 subinterpreter
> >   + RW in all subinterpreters
>
> Isn't that a contradiction? If objects are read-only in all
> subinterpreters (except one), how can they be read/write in all
> subinterpreters?

True.  The two statements, like the rest in the section, are summarizing different details and ideas into which I've been looking.  Several of them are mutually exclusive.

>
>
> All this talk about subinterpreters reminds me of an interesting blog
> post by Armin Ronacher:
>
> http://lucumr.pocoo.org/2014/8/16/the-python-i-would-like-to-see
>

Interesting.  I'd read that before, but not recently.  Armin has some interesting points but I can't say that I agree with his analysis or his conclusions.  Regardless...

> He's quite critical of a number of internal details of the CPython
> interpreter. But what I take from his post is that there could be
> significant advantages to giving the CPython interpreter its own local
> environment, like Lua and Javascript typically do, rather than the
> current model where there is a single process-wide global environment.
> Instead of having multiple subinterpreters all running inside the main
> interpreter, you could have multiple interpreters running in the same
> process, each with their own environment.

But that's effectively the goal!  This proposal will not work if the interpreters are not isolated.  I'm not clear on what Armin thinks is shared between interpreters.  The only consequential shared piece is the GIL and my proposal should render the GIL irrelevant for the most part.

>
> I may be completely misinterpreting things here, but as I understand it,
> this would remove the need for the GIL, allowing even plain old threads
> to take advantage of multiple cores. But that's a separate issue.

If we restrict each subinterpreter to a single thread and are careful with how objects are shared (and sort out exrension modules) then there will be no need for the GIL *within* each subinterpreter.  However there are a couple of things that will keep the GIL around for now.

>
> Armin writes:
>
>     I would like to see an internal interpreter design could be based on
>     interpreters that work independent of each other, with local base
>     types and more, similar to how JavaScript works. This would
>     immediately open up the door again for embedding and concurrency
>     based on message passing. CPUs won't get any faster :)

That's almost exactly what I'm aiming for. :)

-eric