Hello, PyPy list :) I've just started a new list for pirate, a python compiler for the parrot virtual machine. Pirate is written in python and currently uses the standard compiler package. However, as parrot supports native types, I would like to refactor it so that it can also work with PyPy's type annotation system (and possibly integrate with pypy in other ways) The pirate website is here: http://pirate.tangentcode.com/ Quite a bit of work has been done on pirate since I last updated the website back in 2003, mostly by Sam Ruby of http://intertwingly.net/ Sam is giving a python-on-parrot presentation at OSCON on August 4, so I'd like to get a few things cleaned up before then - for example, updating the site, creating a development roadmap, an fixing the code to work with the latest version of parrot, etc. If you're interested in helping out with any of these things, or just want to talk about pirate, then feel free to hop on board. The signup page is here: http://cornerhost.com/mailman/listinfo/pirate Thanks! Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------
Hi Michal, On Sun, Jun 12, 2005 at 11:15:30PM -0400, Michal Wallace wrote:
Interesting, though by no means easy :-) PyPy's type annotation system is not designed at the moment to handle random user code. It can only handle carefully controlled code. From there to a general type annotator for Python, the gap is still large. I may be too deeply buried in the current development work, but at the moment it seems to me that the immediate contribution that PyPy could make to Python-over-Parrot could be different: we could provide the built-in types implementation and standard library modules, which is the necessary complement to the bytecode compiler. Indeed, by tweaking the C code generator, the "standard object space" and the modules could be compiled from PyPy sources to some C code that follows the conventions of Parrot. A bientot, Armin
On Tue, 12 Jul 2005, Armin Rigo wrote:
Hi Armin, Thanks for clarifying. Sounds like my understanding of pypy's current state was completely backwards. There is another approach I've been considering for pirate, which might come in handy for pypy as well: explicitly stating type information in a separate file, along the lines of this type system for scheme: http://www.cs.iastate.edu/~leavens/ComS342-EOPL2e/docs/typedscm_toc.html Basically, you'd have a separate "typesheet" file which said stuff like: # add.ty def add(a=int, b=int): return int which would correspond to the python code: # add.py def add(a, b): return a + b It seems like this would be a very simple way to get type annotation working: we'd just do it by hand. Then once we have real type inference, we could toss it aside. My intuition is that this would be fairly easy for me to add to pypy, but I'm not familiar with the code yet. What do you think? If possible, I'd rather extend your compiler than using the standard compiler module (which is what pirate currently uses)
This is the other part I was backwards on. Just to be clear, and to share this idea with the pirate list, you're saying that all the built-in python types, which you guys have rewritten in your "restricted" python, could be compiled to produce parrot PMCs? Or in other words, that we could write our PMCs in rPython? http://codespeak.net/svn/pypy/dist/pypy/objspace/std/ http://codespeak.net/svn/pypy/dist/pypy/lib/ If I can spend my time hacking on pypy and reap the benefits for pirate, sign me up. I'd rather do that than write PMC's directly. What's the plan for modules like socket, that basically wrap external C routines? Is that in the works for pypy as well? It almost seems like they'd require some extensions to python syntax (sort of like pyrex). Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------
Hi Michal, On Tue, Jul 12, 2005 at 09:58:34PM -0400, Michal Wallace wrote:
Yes, that's still another possible approach: type language design, compiler support, custom bytecodes. Not a solution I'm too fond of :-) but you are welcome to play with it; the future goal of the PyPy compiler is to be flexible enough for this kind of experimentation. (For now it is in development; we have a parser but we directly use the standard library compiler package). Performance-wise, the focus of PyPy is definitely on automatic type inference. Our type inference works nicely already but for "RPython" programs only. RPython is quite difficult to define precisely; ultimately, it's "simple enough to make our type inference happy"... At one point we will try to support user programs that are RPythonic enough, to enable static optimizations on them, but this goes again in the direction of required user awareness. Ultimately, as far as it is practical on a specific target platform, I would rather go for completely transparent just-in-time compilation.
Yes. That's a primary goal of PyPy: write the Python semantics once and compile them to various platforms. Writing a "parrot PMC backend" to generate this C code would be a nice job. That will probably take about the same time as writing the PMCs by hand, and it will probably be more mind-twisting, but then you get for free things like maintainability (tracking Python language changes). To some extent the same is possible with C modules, e.g. regular expressions.
:-)
These modules are handled with a kind of custom language, but which is plain Python code; the types are not specified in the source as with Pyrex, but they are inferred using our type annotation machinery again. For example, os.read() is implemented as: def ll_os_read(fd, count): if count < 0: raise OSError(errno.EINVAL, None) buffer = malloc(STR, count) n = ll_read_into(fd, buffer) if n != count: s = malloc(STR, n) ll_strcpy(s.chars, buffer.chars, n) buffer = s return buffer This code can run normally for testing purposes (malloc() is then a Python function that returns an instance of "_ptr", a class that emulates a pointer to some data). It can also be type-inferenced (starting from the information that 'fd' and 'count' are integers) and translated to C code, where malloc() becomes the real C malloc(). As long as the target platform is C-like, functions like ll_os_read() above can be used to generate code that uses the platform's convention, e.g. parrot's memory management. A bientot, Armin.
Hi Michal, On Sun, Jun 12, 2005 at 11:15:30PM -0400, Michal Wallace wrote:
Interesting, though by no means easy :-) PyPy's type annotation system is not designed at the moment to handle random user code. It can only handle carefully controlled code. From there to a general type annotator for Python, the gap is still large. I may be too deeply buried in the current development work, but at the moment it seems to me that the immediate contribution that PyPy could make to Python-over-Parrot could be different: we could provide the built-in types implementation and standard library modules, which is the necessary complement to the bytecode compiler. Indeed, by tweaking the C code generator, the "standard object space" and the modules could be compiled from PyPy sources to some C code that follows the conventions of Parrot. A bientot, Armin
On Tue, 12 Jul 2005, Armin Rigo wrote:
Hi Armin, Thanks for clarifying. Sounds like my understanding of pypy's current state was completely backwards. There is another approach I've been considering for pirate, which might come in handy for pypy as well: explicitly stating type information in a separate file, along the lines of this type system for scheme: http://www.cs.iastate.edu/~leavens/ComS342-EOPL2e/docs/typedscm_toc.html Basically, you'd have a separate "typesheet" file which said stuff like: # add.ty def add(a=int, b=int): return int which would correspond to the python code: # add.py def add(a, b): return a + b It seems like this would be a very simple way to get type annotation working: we'd just do it by hand. Then once we have real type inference, we could toss it aside. My intuition is that this would be fairly easy for me to add to pypy, but I'm not familiar with the code yet. What do you think? If possible, I'd rather extend your compiler than using the standard compiler module (which is what pirate currently uses)
This is the other part I was backwards on. Just to be clear, and to share this idea with the pirate list, you're saying that all the built-in python types, which you guys have rewritten in your "restricted" python, could be compiled to produce parrot PMCs? Or in other words, that we could write our PMCs in rPython? http://codespeak.net/svn/pypy/dist/pypy/objspace/std/ http://codespeak.net/svn/pypy/dist/pypy/lib/ If I can spend my time hacking on pypy and reap the benefits for pirate, sign me up. I'd rather do that than write PMC's directly. What's the plan for modules like socket, that basically wrap external C routines? Is that in the works for pypy as well? It almost seems like they'd require some extensions to python syntax (sort of like pyrex). Sincerely, Michal J Wallace Sabren Enterprises, Inc. ------------------------------------- contact: michal@sabren.com hosting: http://www.cornerhost.com/ my site: http://www.withoutane.com/ -------------------------------------
Hi Michal, On Tue, Jul 12, 2005 at 09:58:34PM -0400, Michal Wallace wrote:
Yes, that's still another possible approach: type language design, compiler support, custom bytecodes. Not a solution I'm too fond of :-) but you are welcome to play with it; the future goal of the PyPy compiler is to be flexible enough for this kind of experimentation. (For now it is in development; we have a parser but we directly use the standard library compiler package). Performance-wise, the focus of PyPy is definitely on automatic type inference. Our type inference works nicely already but for "RPython" programs only. RPython is quite difficult to define precisely; ultimately, it's "simple enough to make our type inference happy"... At one point we will try to support user programs that are RPythonic enough, to enable static optimizations on them, but this goes again in the direction of required user awareness. Ultimately, as far as it is practical on a specific target platform, I would rather go for completely transparent just-in-time compilation.
Yes. That's a primary goal of PyPy: write the Python semantics once and compile them to various platforms. Writing a "parrot PMC backend" to generate this C code would be a nice job. That will probably take about the same time as writing the PMCs by hand, and it will probably be more mind-twisting, but then you get for free things like maintainability (tracking Python language changes). To some extent the same is possible with C modules, e.g. regular expressions.
:-)
These modules are handled with a kind of custom language, but which is plain Python code; the types are not specified in the source as with Pyrex, but they are inferred using our type annotation machinery again. For example, os.read() is implemented as: def ll_os_read(fd, count): if count < 0: raise OSError(errno.EINVAL, None) buffer = malloc(STR, count) n = ll_read_into(fd, buffer) if n != count: s = malloc(STR, n) ll_strcpy(s.chars, buffer.chars, n) buffer = s return buffer This code can run normally for testing purposes (malloc() is then a Python function that returns an instance of "_ptr", a class that emulates a pointer to some data). It can also be type-inferenced (starting from the information that 'fd' and 'count' are integers) and translated to C code, where malloc() becomes the real C malloc(). As long as the target platform is C-like, functions like ll_os_read() above can be used to generate code that uses the platform's convention, e.g. parrot's memory management. A bientot, Armin.
participants (2)
-
Armin Rigo -
Michal Wallace