[Python-Dev] Gradual migration

[Paul Prescod]

"M.-A. Lemburg" wrote:
> 
> ...
>
> >  * implementation language and Python inheritance semantics are almost
> > completely distinct. After all, we have Python implementations in a
> > variety of languages with minor incompatibilities. Python could have a
> > proper type/class merging even if it were written in assembly language.
> > So let's leave implementation language aside.
> 
> Hey, think of it as opportunity: we can reuse much of C++'s
> optimizations and the integration of Python and C++ applications
> will get *much* easier.

We could make integrating C++ and Python easier through CXX or Py_Cpp.
Perhaps we should ship one of those with Python 2.1. Anyhow, integrating
C++ and Python is not really so hard, in my experience.
 
> A rewrite shouldn't scare anyone away -- much of the existing
> code can be reused since only the Python C APIs of the various
> types will have to be rewritten, not the logic behind the types.

I wasn't really addressing the issue of backwards compatibility of
extensions -- I was talking about Python programs. Nevertheless, I can't
resist:

Porting your app to the Python APIs is often the majority of the work in
a particular extension. A lot of Python extensions consist only of API
glue!
 
> Besides, Py3K will be a project which runs in parallel to the
> 2.x development (at least that's what I've read on some BeOpen
> webpage), so there's really not much to worry about wrt to
> breakage, etc. People will be able to test-drive Py3K while
> using the 2.x series.

I don't see how that helps. If you can't write programs that work both
on the old interpreter and the new one then you need to have a "switch
over" day. The whole point of my doctrine is that Python 3K should run
all code that the version of Python immediately before it did. The most
it can do in terms of breakage is to warn about error messages. In that
case it doesn't matter much whether Python 3K is available at the same
time as 2.X or emerges from Guido's head fully formed.

> >  * there is a hell of a lot we can do to make the type/class split less
> > visible without a major rewrite. For instance, lists could have a
> > __getitem__ method so that they "act like" instances. Instances could
> > return their methods in a dir() so that they "act like" built-in
> > objects. So there is no reason to wait for a complete rewrite to start
> > on this path.
> 
> Right. I didn't want to say that things cannot be done prior
> to the rewrite, only that a rewrite will give us much more
> options that we currently have.

Okay, so do you agree with the rule expressed here:

http://www.python.org/pipermail/python-dev/2000-October/016785.html

> > I don't know what you mean by a "low-level strategy" for classes. It's
> > not like we can somehow use C++ vtables without giving up Python's
> > dynamicity. Python's class handling is about as low-level as it can get
> > in my humble opinion.
> 
> With "low-level" I meant trying to build Python classes and
> instances on top of a very thin layer on top of C++ classes,
> e.g. all slots could be implemented using true C++ methods with
> additional logic to override them using dynamically bound
> Python method objects.

I don't think that there is really interesting magic in a C++ compiler.
A vtable is an array of pointers to functions. We've already implemented
the equivalent in ANSI C. C++ exceptions, constructors, destriuctors and
smart pointers are a little more interesting from a code maintenance and
simplicity point of view. But even then I think that we could get to a
C++ implementation through incremental rewrites.

> > > Differences like the
> > > ones between Unicode and 8-bit strings would then flesh
> > > out as two different subclasses of a generic string type which
> > > again is based on a generic sequence type.
> >
> > Of course that's where we want to go but it doesn't avoid the backwards
> > compatibility problems.
> 
> Huh ? I was talking about clear design... not ways to avoid
> b/w compatibility. 

But the whole point of my original article was backwards
compatibility!!! I didn't address an implementation strategy for Py3K at
all.

> Merging Unicode and strings will hurt one
> way or another. This is simply a consequence of using strings
> as binary containers where Unicode is meant for text only use.

The question I wanted to address is how we can *minimize the pain*. 

> ...
> > C++ does not help. Inheritance does not help. Pluggable compilers do not
> > help. We *will* break code. We just need to decide whether to do it in a
> > thoughtful way that gives people a chance to migrate or put off
> > decisions until the last possible moment and then spring it on them with
> > no between-version upgrade path.
> 
> Just tell Python to use the correct class for what the
> code was written for (and this could be done by plugging in
> a 2.0 compiler). The instances of those classes would still work
> together with other semantics by virtue of exposing the same
> interface, yet the internals would work differently, e.g.
> a module using case insensitive lookup would be compiled using
> case insensitive dictionaries as namespaces.

A mode switch solutiion is fraught with dangers. First there is the
issue of the correct default for the mode switch. 

http://www.python.org/pipermail/python-dev/2000-April/010029.html

Second, there are dangers cutting and pasting between modules.

Anyhow, even if we allow a mode switch we need to be able to help people
to upgrade their modules in a reasonable time. That's what the message I
cited advocates.

>...
> Why spoil all the fun when we haven't even started thinking
> about all the possibilities we could use to make Py3k a
> success ?

Asking for a map of where we are going and how we will get here is
"spoiling all of the fun?" I'm not sure what you are reacting to -- I
didn't advise one implementation strategy or another -- I just said that
we should employ a strategy that minimizes sudden and surprising code
breakage.

-- 
 Paul Prescod - Not encumbered by corporate consensus
Simplicity does not precede complexity, but follows it. 
	- http://www.cs.yale.edu/homes/perlis-alan/quotes.html