[pypy-dev] python 2 and python 3 sharing an interpreter?

[Martijn Faassen]

Hey Armin,

Thanks for your response!

On Fri, Dec 9, 2011 at 12:31 PM, Armin Rigo <arigo at tunes.org> wrote:

> On Thu, Dec 8, 2011 at 20:12, Martijn Faassen <faassen at startifact.com> wrote:
>> Anyway, separate from the binding, I'm wondering about the challenge of two
>> Pythons in one runtime - how hard would it be to generate this with the
>> current PyPy? (imagining two different Python interpreters were available)
>
> Getting two completely separate interpreters in one process is trivial
> in PyPy, quite unlike CPython (blame C).  This works even with the GC
> (you get one for both) and the JIT (you get one JIT that can
> "meta-trace" starting from either interpreter).  That means that maybe
> PyPy is better than CPython as a starting point.

Most definitely, I'd say!

> Of course someone
> still has a serious amount of work to do to integrate the two
> interpreters to the point where it becomes something more than, well,
> two unrelated interpreters in one process.

Yes, that's the challenge. But that sounds like a feasible challenge -
PyPy also has support for transparent proxies, so my hope is that you
could write a Python 2 to Python 3 object proxy and one that does it
the other way around. If you have that and some import namespace
separation hackery you'd already be quite far.

> [ More generally, you can translate several interpreters for different
> languages and getting a single unified JIT, capable of inlining across
> languages.  It is an interesting challenge that explores some unique
> benefits of PyPy --- worth a paper, most probably :-) ]

That is indeed a major benefit. It would allow a much more seamless
upgrade process between multiple incompatible versions of programming
languages. In the Python 2 and Python 3 case, thanks to the unified
JIT you might pay only very little penalty for the integration,
depending on how the JIT and proxies would interact.

Of course all this would have to wait for a Python 3 implementation in
PyPy too. I was unenthusiastic about that before, as I believe PyPy is
in a unique competitive position right now compared to CPython as an
actively developed Python 2 implementation. But having one would make
this integration possible, which I believe would solve endless issues
the community has now with the discontinuity between the two
languages.

In the mean time it should be possible to experiment with just two
Python 2 interpreters in the same process space, and work on a system
to share modules between them. It would not provide any real benefit
to just a single Python 2 as far as I can tell, but it could serve as
a proof of concept.

Is anyone interested in working with me on the latter experiment?

Regards,

Martijn