[pypy-dev] pirate list

[Armin Rigo]

Hi Michal,

On Tue, Jul 12, 2005 at 09:58:34PM -0400, Michal Wallace wrote:
> There is another approach I've been considering
> for pirate, which might come in handy for pypy 
> as well: explicitly stating type information in
> a separate file, along the lines of this type
> system for scheme:

Yes, that's still another possible approach: type language design,
compiler support, custom bytecodes.  Not a solution I'm too fond of :-)
but you are welcome to play with it; the future goal of the PyPy
compiler is to be flexible enough for this kind of experimentation.
(For now it is in development; we have a parser but we directly use the
standard library compiler package).

Performance-wise, the focus of PyPy is definitely on automatic type
inference.  Our type inference works nicely already but for "RPython"
programs only.  RPython is quite difficult to define precisely;
ultimately, it's "simple enough to make our type inference happy"...  At
one point we will try to support user programs that are RPythonic
enough, to enable static optimizations on them, but this goes again in
the direction of required user awareness.  Ultimately, as far as it is
practical on a specific target platform, I would rather go for
completely transparent just-in-time compilation.

> Just to be clear, and to share this idea with the pirate list,
> you're saying that all the built-in python types, which you guys
> have rewritten in your "restricted" python, could be compiled
> to produce parrot PMCs? Or in other words, that we could write
> our PMCs in rPython?

Yes.  That's a primary goal of PyPy: write the Python semantics once and
compile them to various platforms.  Writing a "parrot PMC backend" to
generate this C code would be a nice job.  That will probably take about
the same time as writing the PMCs by hand, and it will probably be more
mind-twisting, but then you get for free things like maintainability
(tracking Python language changes).

To some extent the same is possible with C modules, e.g. regular
expressions.

> If I can spend my time hacking on pypy and reap the benefits
> for pirate, sign me up. I'd rather do that than write PMC's
> directly.

:-)

> What's the plan for modules like socket, that basically
> wrap external C routines? Is that in the works for pypy
> as well? It almost seems like they'd require some 
> extensions to python syntax (sort of like pyrex).

These modules are handled with a kind of custom language, but which is
plain Python code; the types are not specified in the source as with
Pyrex, but they are inferred using our type annotation machinery again.
For example, os.read() is implemented as:

def ll_os_read(fd, count):
    if count < 0:
        raise OSError(errno.EINVAL, None)
    buffer = malloc(STR, count)
    n = ll_read_into(fd, buffer)
    if n != count:
        s = malloc(STR, n)
        ll_strcpy(s.chars, buffer.chars, n)
        buffer = s
    return buffer

This code can run normally for testing purposes (malloc() is then a
Python function that returns an instance of "_ptr", a class that
emulates a pointer to some data).  It can also be type-inferenced
(starting from the information that 'fd' and 'count' are integers) and
translated to C code, where malloc() becomes the real C malloc().

As long as the target platform is C-like, functions like ll_os_read()
above can be used to generate code that uses the platform's convention,
e.g. parrot's memory management.


A bientot,

Armin.