Re: [Python-ideas] API for VM branches (was: PEP 3146)

Collin Winter wrote:

On Tue, Feb 2, 2010 at 2:02 AM, M.-A. Lemburg wrote:

...

Collin Winter wrote: [snip]

...

If such a restrictive plugin-based scheme had been available when we began Unladen Swallow, I do not doubt that we would have ignored it entirely. I do not like the idea of artificially tying the hands of people trying to make CPython faster. I do not see any part of Unladen Swallow that would have been made easier by such a scheme. If anything, it would have made our project more difficult.

I don't think that it has to be restrictive - much to the contrary, it would provide a consistent API to those CPython internals and also clarify the separation between the various parts. Something which currently does not exist in CPython.

We do not need an API to CPython's internals: we are not interfacing with them, we are replacing and augmenting them.

Right, and that's the idea behind making the VM pluggable: you define the boundaries and interfaces of the components you want to replace and then have Python redirect all calls to your implementation by linking against your VM instead of the default one.

...

Note that it may be easier for you (and others) to just take CPython and patch it as necessary. However, this doesn't relieve you from the needed maintenance - which, I presume, is one of the reasons why you are suggesting to merge U-S back into CPython ;-)

That is incorrect. In the year we have been working on Unladen Swallow, we have only updated our vendor branch of CPython 2.6 once, going from 2.6.1 to 2.6.4. We have occasionally cherrypicked patches from the 2.6 maintenance branch to fix specific problems. The maintenance required by upstream CPython changes has been effectively zero.

Patch level releases usually don't include any disruptive changes to the code, so I'm not sure whether your experience with going from 2.6.1 to 2.6.4 provides a good example of what maintaining compatibility with Python's main line of development really means in terms of effort. There are a few things to consider: * if you want to stay compatible with Python, you have to provide updated releases of your version for every supported Python version (including all patch level releases) * you need to build releases with more or less the same frequency as CPython * you need to closely follow Python trunk development if want to avoid lagging behind Python's releases by a few months and all this for a long period of time. Otherwise, I don't see how you could convince users to use your branch instead of the Python mainline version.

We are seeking to merge with CPython for three reasons: 1) verify that python-dev is interested in this project, and that we are not wasting our time; 2) expose the codebase to a wider, more heterogenous testing environment; 3) accelerate development by having more hands on the code. Upstream maintenance requirements have had zero impact on our planning.

I'm sure that python-dev is interested in your project. I'm not sure whether it would be fair to go with your approach instead of the other existing branches, such as Stackless or wpython. How easy would it be for those projects to continue supporting Python if U-S were to be merged into main line Python ?

In any case, I'll be interested in reading your PEP that outlines how the plugin interface should work, which systems will be pluggable, and exactly how Unladen Swallow, WPython and Stackless would benefit. Let's move further discussion of this to python-ideas until there's something more concrete here. The py3k-jit branch will live long enough that we could update it to work with a plugin system, assuming it is demonstrated to be beneficial.

Perhaps I'll write a PEP outlining the idea in more detail. However, without support from developers of such VMs, I don't see much reason in trying to design something that will not eventually get used. I have a feeling that I'm not getting the point across: by making it easier for VM developers to hook into the CPython system, we'd all benefit. VM developers could rely on a consistent interface to develop against, python-dev would know where non-mainline VMs hook into the system and users could choose the VM that best fits their needs. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 03 2010)

...

...

Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/

::: Try our new mxODBC.Connect Python Database Interface for free ! :::: eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/