On Sun, Jun 21, 2015 at 3:54 AM, Antoine Pitrou <solipsis@pitrou.net> wrote:
On Sat, 20 Jun 2015 23:01:20 -0600 Eric Snow <ericsnowcurrently@gmail.com> wrote:
The only consequential shared piece is the GIL and my proposal should render the GIL irrelevant for the most part.
All singleton objects, built-in types are shared and probably a number of other things hidden in dark closets...
Yep. I expect we'll be able to sort those out under the assumption that 99% of the time they can be treated as immutable. We'll then have to find a way to keep the corner cases from breaking the subinterpreter isolation.
Not to mention the memory allocator.
This is a sticky part that I've been considering from almost day 1. It's not the #1 problem to solve, but it will be an important one if we want to have truly parallel subinterpreters.
By the way, what you're aiming to do is conceptually quite similar to Trent's PyParallel (thought Trent doesn't use subinterpreters, his main work is around trying to making object sharing safe without any GIL to trivially protect the sharing), so you may want to pair with him. Of course, you may end up with a Windows-only Python interpreter :-)
Right. I read through Trent's work on several occasions and have gleaned a couple lessons related to object sharing. I was planning on getting in touch with Trent in the near future.
I'm under the impression you're underestimating the task at hand here. Or perhaps you're not and you're just willing to present it in a positive way :-)
I'd like to think it's the latter. :) The main reason why I'm hopeful we can make a meaningful change for 3.6 is that I don't foresee any major changes to CPython's internals. Nearly all the necessary pieces are already there. <handwave/> I'm also intent on taking a minimal approach initially. We can build on it from there, easing restrictions that allowed us to roll out the initial implementation more quickly. All that said, I won't be surprised if it takes the entire 3.6 dev cycle to get it right. -eric