[Python-ideas] A python bridge between versions

[Chris Angelico]

On Fri, Feb 28, 2014 at 10:26 AM, ian o <ian.team.python at gmail.com> wrote:
> Ptyhon can call C. C can call python. So python3 call C which calls python2.

That's all very well if all you want to do is execute code. But as
soon as you want to transfer data across, you have to do a whole lot
of careful rummaging.

First off, you'll need to have some C-level module that imports a Py2
module, identifies all its exported symbols (technically all its
symbols, but you could save some effort by just looking at __all__ and
most people won't see the difference), and creates a buffer layer
between that and Py3: whenever someone calls up one of those symbols,
create the translation and send it along. You can't have the two
interpreters sharing GC state.

But second, you have to deal with the fundamental data type
differences. It may seem obvious enough - a function call goes through
C and eventually becomes a function call, and so on - but the two
interpreters fundamentally differ. Start with the easy one: int versus
long. Py2 has two integer types; Py3 dropped int and renamed long to
int. You need to traverse that gap.

And then the big one. The really really big one. Bytes versus Unicode.
How are you going to share strings across the boundary? Will all Py2's
quoted strings come out as bytes in Py3? Will you encode and decode
across the boundary (and if so, what encoding?)?

It's fundamentally *hard* to do this. And there's another problem, too
- a philosophical one. Let's suppose you put in all this work to make
spamify.py available under Python 3. Your application, which needs
spamify, can now migrate; but where's the impetus for spamify itself
to start supporting Py3? Now that this shim is available, they can
just carry on forever, right? So as soon as you create this, you
effectively doom yourself to indefinite support, which is a *lot* of
work. A new version of spamify might add a new feature, which you then
need to support. Or it might start using a different encoding for its
strings, and suddenly breaking everything. You have to keep up with
all of that, *and* you're encouraging libraries to stay on Py2. In
effect, you're encouraging applications to move to Py3, but
encouraging libraries to stay on Py2.

Personally, I'd rather people lean on their library creators to start
supporting Py3. In the long run, it's going to be less work anyway.
The only person who truly knows how a module should translate Py2
strings into Py3 strings is that module's author.

ChrisA