Self hosting Python? (was: Python is just as good as C++ for real apps)

[Kragen Sitaker]

russell_turpin at hotmail.com (Russell Turpin) writes:
> Courageous <jkraska at san.rr.com> wrote in message news:<b3as4ushqd8rni7389puglkk6hickhvgp5 at 4ax.com>...
> > Why isn't Python self-hosting?
> 
> I wonder how many Lisp systems are fully self-hosted?

A lot of them; I think all the commercial Common Lisps, plus CMUCL.
Bruno Haible's Clisp is an exception.

Squeak occupies an interesting middle ground; the Squeak interpreter,
a fairly traditional bytecode interpreter, is written in (a restricted
subset of) Squeak, and there's a compiler written in Squeak that
compiles the interpreter into C.  This makes it easy to port to new
architectures.

> Self hosting is pretty easy with languages that include a low level
> model of the machine, such as C. I have no experience doing this for
> higher-level languages. My first suspicion -- which may be wrong --
> is that 100% self-hosting would be difficult to do *efficiently*
> without using a memory management library written in a different
> language, or hand-translation of the some of the Python code. The
> problem is that Python simply doesn't give you a way to express
> memory management at the machine level. Everything is a reference to
> an object. There is no way to code an "array of bytes," because that
> notion simply isn't in the language. That's very good for everything
> .. EXCEPT byte management.

It's not hard to add unsafe operations to a safe language, and that
lets you do your memory management (code generation, library loading,
etc.) in the otherwise-safe language.

CMUCL does this, and as a result, when you want to hook CMUCL up to a
new C library, you can write your glue code in Lisp instead of C.
This is a really big advantage.  I wish I could do that in Python.

> This all seems very reasonable to me. There is NO reason that an
> application language should provide the constructs for modeling the
> machine. Let's do the things that require such low-level notions in
> C, including standard memory management and implementing numeric
> packages. Removing it from the domain of applications is a boon.

I agree that applications normally shouldn't use those things, but you
know, I'd just much rather write a Python compiler in Python than in
C.

An application language should provide the constructs its applications
need to use.  An application language for numerical scientific
computing applications should provide multidimensional rectangular
homogeneous arrays.  An application language for compiling code on the
fly should provide instructions and memory management.

> Python isn't just as good as C++ for real applications -- it is far
> better.

I'm interested to hear more.

> That said, it might be worthwhile to encapsulate the memory 
> management (and other low-level mechanisms required) behind 
> a fixed interface, implemented in C, and then make a Python 
> compiler in Python.

Once you have the Python compiler, of course, the fixed interface can
be implemented in Python instead of C.

> If anyone wants to pay me to do this, I'm eager to jump into the
> internals of the language!

I'll let you know if I get rich and have time to visit Austin to work
with you on it; it sounds like a fun project to me too.