[Python-Dev] Gradual migration

[M.-A. Lemburg]

Paul Prescod wrote:
> 
> "M.-A. Lemburg" wrote:
> >
> >...
> >
> > I think will simply be a consequence of doing a complete rewrite
> > of the interpreter for Py3K. AFAIR, the only truely feasable
> > solution would be doing the rewrite in a widely portable
> > subset of C++ and then managing classes at that level.
> 
> I disagree for a variety of reasons:
> 
>  * implementation language and Python inheritance semantics are almost
> completely distinct. After all, we have Python implementations in a
> variety of languages with minor incompatibilities. Python could have a
> proper type/class merging even if it were written in assembly language.
> So let's leave implementation language aside.

Hey, think of it as opportunity: we can reuse much of C++'s 
optimizations and the integration of Python and C++ applications
will get *much* easier.
 
A rewrite shouldn't scare anyone away -- much of the existing
code can be reused since only the Python C APIs of the various
types will have to be rewritten, not the logic behind the types.

Besides, Py3K will be a project which runs in parallel to the
2.x development (at least that's what I've read on some BeOpen
webpage), so there's really not much to worry about wrt to
breakage, etc. People will be able to test-drive Py3K while
using the 2.x series.

>  * there is a hell of a lot we can do to make the type/class split less
> visible without a major rewrite. For instance, lists could have a
> __getitem__ method so that they "act like" instances. Instances could
> return their methods in a dir() so that they "act like" built-in
> objects. So there is no reason to wait for a complete rewrite to start
> on this path.

Right. I didn't want to say that things cannot be done prior
to the rewrite, only that a rewrite will give us much more
options that we currently have.

>  * It may even be the case that we can get from here to complete merged
> type/class semantics WITHOUT a rewrite. If a mad genious had not written
> stackless Python I would have sworn up and down that stackless Python
> would require a Python rewrite. It didn't. If another slightly less mad
> genious had not integrated ints and longs I would never have believed it
> possible without another rewrite. Someone needs to do an analysis of
> what it takes to merge types and classes and *demonstrate* that it
> requires a rewrite rather than *asserting* that it requires a rewrite.
>
> In other words, let's stop sprinkling the "major rewrite" pixie dust on
> hard problems and instead decide where we want to go and how we want to
> get there!

See above.
 
> > Moving to a common and very low-level strategy for classes
> > will allows us to put even the most basic types (strings and
> > numbers) into an inheritence tree.
> 
> I don't know what you mean by a "low-level strategy" for classes. It's
> not like we can somehow use C++ vtables without giving up Python's
> dynamicity. Python's class handling is about as low-level as it can get
> in my humble opinion.

With "low-level" I meant trying to build Python classes and
instances on top of a very thin layer on top of C++ classes,
e.g. all slots could be implemented using true C++ methods with
additional logic to override them using dynamically bound
Python method objects.

> > Differences like the
> > ones between Unicode and 8-bit strings would then flesh
> > out as two different subclasses of a generic string type which
> > again is based on a generic sequence type.
> 
> Of course that's where we want to go but it doesn't avoid the backwards
> compatibility problems.

Huh ? I was talking about clear design... not ways to avoid
b/w compatibility. Merging Unicode and strings will hurt one
way or another. This is simply a consequence of using strings
as binary containers where Unicode is meant for text only use.

> We can do this today using proxying.

Could you explain this ?
 
> > The same could be done for dictionaries: special ones for
> > just string keys, case insensitive lookups, etc. could all
> > be subclasses of a generic mapping class.
> 
> We can easily do this today.

No we can't: Python's use of pointer compares to find out
which type it is dealing with prevent this.
 
> > Dito for numbers (and division strategies).
> 
> There's no way I'm going to let you get away with that little sleight of
> hand. How is inheritance holy water going to allow us to change the
> semantics of:
> 
> 5/2
> 
> without breaking code??
> 
> The same goes for
> 
> a.b = 5
> 
> versus
> 
> a.B = 5
> 
> C++ does not help. Inheritance does not help. Pluggable compilers do not
> help. We *will* break code. We just need to decide whether to do it in a
> thoughtful way that gives people a chance to migrate or put off
> decisions until the last possible moment and then spring it on them with
> no between-version upgrade path.

Just tell Python to use the correct class for what the
code was written for (and this could be done by plugging in
a 2.0 compiler). The instances of those classes would still work
together with other semantics by virtue of exposing the same
interface, yet the internals would work differently, e.g.
a module using case insensitive lookup would be compiled using
case insensitive dictionaries as namespaces.

> > By following this principle there won't be all that much
> > breakage, since the old functionality will still be around,
> > only the defaults will have changed.
> 
> When you change defaults you break code. Keeping the old functionality
> around is barely helpful.  Nobody has EVER proposed a change to Python
> where something that was previously possible is now made impossible. So
> whatever our strategy, the PROBLEM is changing defaults. The solution is
> telling people what defaults are changing in what timeline and
> discouraging them from depending on the old defaults.

All true. I was just referring to the possibility of keeping
the old semantics around in case some module relies on them.
In this ideal world, a simple "define pythonlevel=2.0"
would suffice to make the old module work with Py3k.
 
> > Add to this pluggable compilers and ceval loops, plus a nice
> > way of configuring the lot on a per-module basis and you're
> > set. (Ok, it's a fluffy clouds image, but you get the picture ;-)
> 
> Sounds mythical. I'm trying to take it out of the realm of fluffy clouds
> and bring it into the world that people can plan their businesses and
> coding strategies around.

Hmm, wasn't Py3k meant as sandbox for new experiments ? 
The 2.x series is for doing business with, IMHO at least. 

At the current dev speed we'll have plenty of time to get Py3k
rock solid. Then we can add all the backward compatibility layers
we want to it. If the design is a good, adding these layers
won't pose much of a problem.

Why spoil all the fun when we haven't even started thinking
about all the possibilities we could use to make Py3k a
success ?

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/