[Cython] Rewriting/compiling parts of CPython's stdlib in Cython

[Robert Bradshaw]

On Fri, Mar 25, 2011 at 6:42 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Sturla Molden, 25.03.2011 14:03:
>>
>> Den 24.03.2011 20:38, skrev Robert Bradshaw:
>>>
>>> I started a list at http://wiki.cython.org/Unsupported . I'd say we
>>> can be as compatible as Jython/IronPython is, and more than CPython is
>>> between minor versions. I would be happy with a short, well-justified
>>> list of differences. This will be clearer once the community (which
>>> we're a part of) defines what Python vs. implementation details means.
>>
>> Looking at Guido's comment, Cython must be able to compile all valid
>> Python if this will have any chance of success.

Good thing that's our goal (pending an actual definition of "all valid Python.")

> I think there are two levels of required compatibility, the lower one of
> which applies to a reimplementation of C modules in Cythen. That's the
> reason why I see that as the more worthwhile goal for now.
>
>
>> Is the plan to include Cython in the standard library? I don't think a
>> large external dependency like Cython will be accepted unless it's a part
>> of the CPython distribution.
>
> It was the plan a while ago, but the problem is that Cython's evolution (and
> stability) isn't very much in line with that of CPython. I'm not as
> pessimistic as you seem to be, though. From the POV of CPython, Cython is
> basically just a build tool. We could try to come up with a release series
> that we consider stable enough to base CPython on it. If we agree to
> maintain that for the whole lifetime of the corresponding CPython
> release(s), it wouldn't necessarily have to be part of CPython itself
> (although it could...)

As a first step, CPython would just ship the generated C, only
requiring Cython as a development dependency, not a build dependency.
This would also be safer from the stability POV.

>> Why stop with the standard library? Why not implement the whole CPython
>> interpreter in Cython?
>
> No-one said we'd stop there. ;)

Of course the CPython interpreter is a large, fairly-optimized C
codebase already, so the pitfalls of just re-writing it for the sake
of re-writing it probably outweigh the gains.

> However, going down that route will quickly turn into a problem of
> boot-strapping. How would you run Cython without a Python interpreter? Would
> you need an old interpreter installed in order to run Cython to build a new
> one? Currently, you only need a C compiler and cmmi tools. I could easily
> understand anyone who'd reject entering into such a recursive dependency.

It would be as easy as bootstrapping gcc... :).

- Robert