[Python-ideas] A python bridge between versions

[James Powell]

On 02/27/2014 06:51 PM, M.-A. Lemburg wrote:
> On 28.02.2014 00:26, ian o wrote:
>>  [... Embed Python 2 in Python 3 ...]
>>
>>  Thoughts or criticisms?

> There's a catch here: Python 2 and Python 3 use the same C APIs,
> so you'd have to separate the two in some way to make both live
> in the same process.

I actually have a working embedding of Python 3 into Python 2, which
I've presented at a few conferences, most recently at PyData London this
past weekend.

It's just a C-extension module that embeds a Python 3
PyRun_SimpleString. I haven't gotten around to building a shim to
interact with Python 3 objects in Python 2 (and this would require a
little bit of sophistication to handle GIL, GC, &c. issues.)

Still, it's a working example of Python 2 and Python 3 running in the
same process.

> It's not impossible, but it can potentially ruin the idea, since
> C extensions for both Python versions will have to link the right
> set of C APIs.

I did this by "source filtering" - i.e., rewriting exported symbols to
allow linking against both interpreters. This is admittedly a very
primitive approach and limits its use in production.

I recently put some time into trying to redo this embedding using cffi
and worked through a few problems in this approach with one of the
cython/numba developers. I haven't gotten anything working yet. I may
need to ask Xzibit if he has any suggestions.

I don't know if this idea has any serious applications. It's definitely
been a fun toy for presenting ideas about interpreter embedding and
exploring certain facets of CPython!

Cheers,
James Powell

follow: @dontusethiscode + @nycpython
attend: nycpython.org
read:   seriously.dontusethiscode.com