From what I see on the pyjitpl5 branch, jit support seems to be wildly in flux at the moment. I'm thinking I probably want to put jit support on hold until things settle down, although it's definitely on my todo
I have built an RPython implementation of befunge98 on top of pypy. It took about a week at 30%, and about half of that was spent writing a befunge implementation of pidigits[1]. I honestly expected to have more trouble getting this working, which is one of the reasons I chose something as simple as befunge. There were some hiccups, certainly, but compared to working with parrot and llvm, pypy was startlingly easy to get up and running. On to benchmarks. Time taken to compute the first 1,000 digits of pi: befunge.py: 3:03.18 befunge-c: 0:02.76 As advertised, I was able to write and test in python and run it through the translator to get... the wrong test output. This was, naturally, my own fault. As it turns out, pidigits requires arbitrary precision integer support. A quick skim of rarithmatic and rbignum and a bit of hacking later, and I got the awesomely cool results you see above. It was my plan at this point to lay down some benchmarks against other language implementations of pidigits showing how awesome pypy is. Unfortunately, the pidigits test is basically entirely limited by the performance of the bignum library in use. This effectively means that every language that can use gmp does, and gets a result of 3-7 seconds, which makes the pidigits test basically useless for meaningful interpreter benchmarking. That said, it is still a good benchmark for bignum libraries, so here goes: Time taken to compute the first 10,000 digits of pi: befunge-c: 5:22.13 pypy-c [2]: 5:35.26 python-2.5 [2]: 1:31.56 python-2.5(gmp) [3]: 0:10.22 list. Supporting jit on top of befunge will, I think, be an interesting edge case because the language was specifically designed to be hard to compile. Does pypy want a befunge interpreter? It is a rather easier to read interpreter than malbolge or javascript, although I did write it myself, so I may be biased. Some of the more esoteric instructions and edge cases are still missing, and there are several improvements that I would still like to try, but it basically works now, so I figured I would throw it out there. Pypy isn't using a DVCS, so I'm not sure what the best way is for me to share this code. For now I've just attached it inline -- it's only 12KiB, packed up. Untar it into the lang directory and it will create a befunge subdirectory. Let me know if there is a better way to share code. -Terrence Cole [1] - http://shootout.alioth.debian.org/u32q/benchmark.php?test=pidigits&lang=all#about [2] - http://shootout.alioth.debian.org/u32q/benchmark.php?test=pidigits&lang=python&id=1 [3] - http://shootout.alioth.debian.org/u32q/benchmark.php?test=pidigits&lang=python&id=3
Hi. It's really cool that you find pypy as a good platform for writing interpreters, we're definitely happy with that :-) In general, we try to remove stuff from lang rather then put more of it there. So js/smalltalk/befunge interpreters would hopefully soon become projects on its own. On the other hand, we would be happy to give you commit access on codespeak so you can check stuff in somewhere there. The exact directory layout where to put other interpreters (and how to make different pages for them etc.) is yet to be determined. In case you want to use some other hosting service, like google code, feel free to do so as well. We would be more than happy to reference it from our web page. Cheers, fijal PS. Everybody is more or less on holiday right now, at least until next week, hence late response, sorry for that. PPS. If you check this stuff somewhere, I think it makes sense to mention it on the pypy blog, so please keep us updated. On Sat, Aug 8, 2009 at 5:26 PM, Terrence Cole<list-sink@trainedmonkeystudios.org> wrote:
I have built an RPython implementation of befunge98 on top of pypy. It took about a week at 30%, and about half of that was spent writing a befunge implementation of pidigits[1]. I honestly expected to have more trouble getting this working, which is one of the reasons I chose something as simple as befunge. There were some hiccups, certainly, but compared to working with parrot and llvm, pypy was startlingly easy to get up and running.
On to benchmarks.
Time taken to compute the first 1,000 digits of pi: befunge.py: 3:03.18 befunge-c: 0:02.76
As advertised, I was able to write and test in python and run it through the translator to get... the wrong test output. This was, naturally, my own fault. As it turns out, pidigits requires arbitrary precision integer support. A quick skim of rarithmatic and rbignum and a bit of hacking later, and I got the awesomely cool results you see above.
It was my plan at this point to lay down some benchmarks against other language implementations of pidigits showing how awesome pypy is. Unfortunately, the pidigits test is basically entirely limited by the performance of the bignum library in use. This effectively means that every language that can use gmp does, and gets a result of 3-7 seconds, which makes the pidigits test basically useless for meaningful interpreter benchmarking. That said, it is still a good benchmark for bignum libraries, so here goes:
Time taken to compute the first 10,000 digits of pi: befunge-c: 5:22.13 pypy-c [2]: 5:35.26 python-2.5 [2]: 1:31.56 python-2.5(gmp) [3]: 0:10.22
From what I see on the pyjitpl5 branch, jit support seems to be wildly in flux at the moment. I'm thinking I probably want to put jit support on hold until things settle down, although it's definitely on my todo list. Supporting jit on top of befunge will, I think, be an interesting edge case because the language was specifically designed to be hard to compile.
Does pypy want a befunge interpreter? It is a rather easier to read interpreter than malbolge or javascript, although I did write it myself, so I may be biased. Some of the more esoteric instructions and edge cases are still missing, and there are several improvements that I would still like to try, but it basically works now, so I figured I would throw it out there. Pypy isn't using a DVCS, so I'm not sure what the best way is for me to share this code. For now I've just attached it inline -- it's only 12KiB, packed up. Untar it into the lang directory and it will create a befunge subdirectory. Let me know if there is a better way to share code.
-Terrence Cole
[1] - http://shootout.alioth.debian.org/u32q/benchmark.php?test=pidigits&lang=all#about [2] - http://shootout.alioth.debian.org/u32q/benchmark.php?test=pidigits&lang=python&id=1 [3] - http://shootout.alioth.debian.org/u32q/benchmark.php?test=pidigits&lang=python&id=3
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
On Tue, 2009-08-11 at 03:16 -0600, Maciej Fijalkowski wrote:
Hi.
It's really cool that you find pypy as a good platform for writing interpreters, we're definitely happy with that :-) In general, we try to remove stuff from lang rather then put more of it there. So js/smalltalk/befunge interpreters would hopefully soon become projects on its own.
Thank you for the reply. I was thinking that the current situation in the lang directory would be unmanagable and I was wondering what I was missing :-). I have not run into any documentation on making the pypy toolchain run from an installed location, rather than from the svn checkout dir. Since I don't even understand how autopath works yet, I figured that getting something more sophisticated working than what everyone else is doing in the lang directory would be an adventure better left for later. Is there a plan for making pypy installable? Is it already possible and I just haven't looked hard enough?
On the other hand, we would be happy to give you commit access on codespeak so you can check stuff in somewhere there. The exact directory layout where to put other interpreters (and how to make different pages for them etc.) is yet to be determined.
That project would be more worthy of commit access than a befunge interpreter, I think. Maybe I should go look at how py's installer works.
In case you want to use some other hosting service, like google code, feel free to do so as well. We would be more than happy to reference it from our web page.
I'll check out my options when I get some freetime. If pypy gets good support for hosting language implementations worked out at some point in the future, we can always move it.
Cheers, fijal
PS. Everybody is more or less on holiday right now, at least until next week, hence late response, sorry for that.
PPS. If you check this stuff somewhere, I think it makes sense to mention it on the pypy blog, so please keep us updated.
Will do! -Terrence
On Sat, Aug 8, 2009 at 5:26 PM, Terrence Cole<list-sink@trainedmonkeystudios.org> wrote:
I have built an RPython implementation of befunge98 on top of pypy. It took about a week at 30%, and about half of that was spent writing a befunge implementation of pidigits[1]. I honestly expected to have more trouble getting this working, which is one of the reasons I chose something as simple as befunge. There were some hiccups, certainly, but compared to working with parrot and llvm, pypy was startlingly easy to get up and running.
On to benchmarks.
Time taken to compute the first 1,000 digits of pi: befunge.py: 3:03.18 befunge-c: 0:02.76
As advertised, I was able to write and test in python and run it through the translator to get... the wrong test output. This was, naturally, my own fault. As it turns out, pidigits requires arbitrary precision integer support. A quick skim of rarithmatic and rbignum and a bit of hacking later, and I got the awesomely cool results you see above.
It was my plan at this point to lay down some benchmarks against other language implementations of pidigits showing how awesome pypy is. Unfortunately, the pidigits test is basically entirely limited by the performance of the bignum library in use. This effectively means that every language that can use gmp does, and gets a result of 3-7 seconds, which makes the pidigits test basically useless for meaningful interpreter benchmarking. That said, it is still a good benchmark for bignum libraries, so here goes:
Time taken to compute the first 10,000 digits of pi: befunge-c: 5:22.13 pypy-c [2]: 5:35.26 python-2.5 [2]: 1:31.56 python-2.5(gmp) [3]: 0:10.22
From what I see on the pyjitpl5 branch, jit support seems to be wildly in flux at the moment. I'm thinking I probably want to put jit support on hold until things settle down, although it's definitely on my todo list. Supporting jit on top of befunge will, I think, be an interesting edge case because the language was specifically designed to be hard to compile.
Does pypy want a befunge interpreter? It is a rather easier to read interpreter than malbolge or javascript, although I did write it myself, so I may be biased. Some of the more esoteric instructions and edge cases are still missing, and there are several improvements that I would still like to try, but it basically works now, so I figured I would throw it out there. Pypy isn't using a DVCS, so I'm not sure what the best way is for me to share this code. For now I've just attached it inline -- it's only 12KiB, packed up. Untar it into the lang directory and it will create a befunge subdirectory. Let me know if there is a better way to share code.
-Terrence Cole
[1] - http://shootout.alioth.debian.org/u32q/benchmark.php?test=pidigits〈=all#about [2] - http://shootout.alioth.debian.org/u32q/benchmark.php?test=pidigits〈=python&id=1 [3] - http://shootout.alioth.debian.org/u32q/benchmark.php?test=pidigits〈=python&id=3
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
On Wed, 12 Aug 2009 17:44:02 -0700 Terrence Cole <list-sink@trainedmonkeystudios.org> wrote:
On Tue, 2009-08-11 at 03:16 -0600, Maciej Fijalkowski wrote:
Hi.
It's really cool that you find pypy as a good platform for writing interpreters, we're definitely happy with that :-) In general, we try to remove stuff from lang rather then put more of it there. So js/smalltalk/befunge interpreters would hopefully soon become projects on its own.
Thank you for the reply. I was thinking that the current situation in the lang directory would be unmanagable and I was wondering what I was missing :-).
I have not run into any documentation on making the pypy toolchain run from an installed location, rather than from the svn checkout dir. Since I don't even understand how autopath works yet, I figured that getting something more sophisticated working than what everyone else is doing in the lang directory would be an adventure better left for later. Is there a plan for making pypy installable? Is it already possible and I just haven't looked hard enough?
What I'm currently doing with an interpreter I'm messing around with is to just put the path to PyPy in my PYTHONPATH (I'm using the 1.1.0 release right now, but I think it would work for an svn checkout as well). That works for running on top of CPython, even with my interpreter project directory completely outside of the PyPy tree. Translating is a little less satisfactory. I have a makefile with the following: # Defualt place to look for PyPy being installed. PYPY_DIR := ~/pypy-1.1.0/ # Set target-specific variable to set the translation backend based on the name # of the target, e.g. fundy-jvm uses the jvm backend. fundy-%: BACKEND = --backend=$(@:fundy-%=%) # Default target: fundy translated with the C backend. .PHONY: default default: fundy-c fundy-%: *.py fundy.grammar python $(PYPY_DIR)/pypy/translator/goal/translate.py $(OPT) $(BACKEND) $(BATCH) $(TRANSLATE_FLAGS) target_fundy (plus a few default settings for the variables I haven't defined here, but they're pretty trivial), and a target_fundy.py file that provides the target(driver, args) function. Then I can just type make, or make fundy-jvm, etc, to get the interpreter translated for any backend (although the C one is the only one working on my system). I started with this project in the lang directory of a PyPy svn checkout, but when the 1.1.0 release was made and I ended up using my interpreter as the basis for a university essay on programming languages, it spurred me to figure out how to get it working as a more independent project. I didn't find much information in the PyPy docs on how to do this, I just hacked it together. If there are better ways of going about it, I'd like to know! I do have ~/pypy-1.1.0/pypy/bin in my PATH (translatorshell.py is just too nifty), and I have thought it would be handy if there were an entry point executable for the translation process in there as well. If translate.py or something equivalent was there, then installing PyPy would consist of untarring it, setting PYTHONPATH and PATH, and then interpreter projects could just use it a little more easily. I guess there's a difference between people who are installing the PyPy python interpreter, and those who are installing the PyPy translation toolchain. But what the docs say now about how to build things with the PyPy toolchain is to cd to pypy/bin/translator/goal and run translate.py targetfile, which looks a bit like to make things able to be translated you have to put a targetfile there (and it has to be able to import the rest of your project, which means either your project is in the PyPy tree, or it's on your PYTHONPATH). -- Ben
On Thu, 2009-08-13 at 12:27 +1000, Ben Mellor wrote:
On Wed, 12 Aug 2009 17:44:02 -0700 Terrence Cole <list-sink@trainedmonkeystudios.org> wrote:
On Tue, 2009-08-11 at 03:16 -0600, Maciej Fijalkowski wrote:
Hi.
It's really cool that you find pypy as a good platform for writing interpreters, we're definitely happy with that :-) In general, we try to remove stuff from lang rather then put more of it there. So js/smalltalk/befunge interpreters would hopefully soon become projects on its own.
Thank you for the reply. I was thinking that the current situation in the lang directory would be unmanagable and I was wondering what I was missing :-).
I have not run into any documentation on making the pypy toolchain run from an installed location, rather than from the svn checkout dir. Since I don't even understand how autopath works yet, I figured that getting something more sophisticated working than what everyone else is doing in the lang directory would be an adventure better left for later. Is there a plan for making pypy installable? Is it already possible and I just haven't looked hard enough?
What I'm currently doing with an interpreter I'm messing around with is to just put the path to PyPy in my PYTHONPATH (I'm using the 1.1.0 release right now, but I think it would work for an svn checkout as well). That works for running on top of CPython, even with my interpreter project directory completely outside of the PyPy tree.
That did the trick! My test suite is a bit of a kludge now though since I can't reference my interpreter from an absolute pypy path. I have added .. to my sys.path manually to get them working again, but it's very ugly. Is there a better way to handle this?
Translating is a little less satisfactory. I have a makefile with the following:
# Defualt place to look for PyPy being installed. PYPY_DIR := ~/pypy-1.1.0/
# Set target-specific variable to set the translation backend based on the name # of the target, e.g. fundy-jvm uses the jvm backend. fundy-%: BACKEND = --backend=$(@:fundy-%=%)
# Default target: fundy translated with the C backend. .PHONY: default default: fundy-c
fundy-%: *.py fundy.grammar python $(PYPY_DIR)/pypy/translator/goal/translate.py $(OPT) $(BACKEND) $(BATCH) $(TRANSLATE_FLAGS) target_fundy
(plus a few default settings for the variables I haven't defined here, but they're pretty trivial), and a target_fundy.py file that provides the target(driver, args) function.
Then I can just type make, or make fundy-jvm, etc, to get the interpreter translated for any backend (although the C one is the only one working on my system).
I started with this project in the lang directory of a PyPy svn checkout, but when the 1.1.0 release was made and I ended up using my interpreter as the basis for a university essay on programming languages, it spurred me to figure out how to get it working as a more independent project.
I didn't find much information in the PyPy docs on how to do this, I just hacked it together. If there are better ways of going about it, I'd like to know!
I suppose using PYTHONPATH should be obvious, in retrospect. Usually, when I have these sorts of problems, I create a setup.py and just get everything installed into site-packages. In fact, I do think that having pypy in an installed location would be more satisfying than messing with the path, but I suppose this works for now.
I do have ~/pypy-1.1.0/pypy/bin in my PATH (translatorshell.py is just too nifty), and I have thought it would be handy if there were an entry point executable for the translation process in there as well. If translate.py or something equivalent was there, then installing PyPy would consist of untarring it, setting PYTHONPATH and PATH, and then interpreter projects could just use it a little more easily.
Agreed. It would be very nice to have a pypy.translate program in /usr/bin. There are some (non-interpreter) projects I would consider writing in RPython just for the performance benefits of translation.
I guess there's a difference between people who are installing the PyPy python interpreter, and those who are installing the PyPy translation toolchain.
Very true. If all you want is pypy-*, how much more do you need installed than the executable and the modules directory?
But what the docs say now about how to build things with the PyPy toolchain is to cd to pypy/bin/translator/goal and run translate.py targetfile, which looks a bit like to make things able to be translated you have to put a targetfile there (and it has to be able to import the rest of your project, which means either your project is in the PyPy tree, or it's on your PYTHONPATH).
That was exactly my thinking as well.
-- Ben
Thank you for the help. That was exactly what I needed and I'm very glad I did not have to go blundering about relearning all of that for myself. - Terrence
_______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
On Fri, Aug 14, 2009 at 3:44 PM, Terrence Cole<list-sink@trainedmonkeystudios.org> wrote:
On Thu, 2009-08-13 at 12:27 +1000, Ben Mellor wrote:
On Wed, 12 Aug 2009 17:44:02 -0700 Terrence Cole <list-sink@trainedmonkeystudios.org> wrote:
On Tue, 2009-08-11 at 03:16 -0600, Maciej Fijalkowski wrote:
Hi.
It's really cool that you find pypy as a good platform for writing interpreters, we're definitely happy with that :-) In general, we try to remove stuff from lang rather then put more of it there. So js/smalltalk/befunge interpreters would hopefully soon become projects on its own.
Thank you for the reply. I was thinking that the current situation in the lang directory would be unmanagable and I was wondering what I was missing :-).
I have not run into any documentation on making the pypy toolchain run from an installed location, rather than from the svn checkout dir. Since I don't even understand how autopath works yet, I figured that getting something more sophisticated working than what everyone else is doing in the lang directory would be an adventure better left for later. Is there a plan for making pypy installable? Is it already possible and I just haven't looked hard enough?
What I'm currently doing with an interpreter I'm messing around with is to just put the path to PyPy in my PYTHONPATH (I'm using the 1.1.0 release right now, but I think it would work for an svn checkout as well). That works for running on top of CPython, even with my interpreter project directory completely outside of the PyPy tree.
That did the trick! My test suite is a bit of a kludge now though since I can't reference my interpreter from an absolute pypy path. I have added .. to my sys.path manually to get them working again, but it's very ugly. Is there a better way to handle this?
I'm not sure, but PYTHONPATH can handle multiple paths, like: PYTHONPATH=path_a:path_b is this what you wanted? Cheers, fijal
On Mon, 2009-08-17 at 14:51 -0600, Maciej Fijalkowski wrote:
On Fri, Aug 14, 2009 at 3:44 PM, Terrence Cole<list-sink@trainedmonkeystudios.org> wrote:
On Thu, 2009-08-13 at 12:27 +1000, Ben Mellor wrote:
On Wed, 12 Aug 2009 17:44:02 -0700 Terrence Cole <list-sink@trainedmonkeystudios.org> wrote:
On Tue, 2009-08-11 at 03:16 -0600, Maciej Fijalkowski wrote:
Hi.
It's really cool that you find pypy as a good platform for writing interpreters, we're definitely happy with that :-) In general, we try to remove stuff from lang rather then put more of it there. So js/smalltalk/befunge interpreters would hopefully soon become projects on its own.
Thank you for the reply. I was thinking that the current situation in the lang directory would be unmanagable and I was wondering what I was missing :-).
I have not run into any documentation on making the pypy toolchain run from an installed location, rather than from the svn checkout dir. Since I don't even understand how autopath works yet, I figured that getting something more sophisticated working than what everyone else is doing in the lang directory would be an adventure better left for later. Is there a plan for making pypy installable? Is it already possible and I just haven't looked hard enough?
What I'm currently doing with an interpreter I'm messing around with is to just put the path to PyPy in my PYTHONPATH (I'm using the 1.1.0 release right now, but I think it would work for an svn checkout as well). That works for running on top of CPython, even with my interpreter project directory completely outside of the PyPy tree.
That did the trick! My test suite is a bit of a kludge now though since I can't reference my interpreter from an absolute pypy path. I have added .. to my sys.path manually to get them working again, but it's very ugly. Is there a better way to handle this?
I'm not sure, but PYTHONPATH can handle multiple paths, like: PYTHONPATH=path_a:path_b
is this what you wanted?
I think it would work, but maybe I should just describe what I'm trying to do. My tests are in befunge/tests/. I need to import the interpreter from befunge/Interpreter.py. Before, I just dropped autopath.py into tests and did 'import pypy.lang.befunge.Interpreter'. Now that autopath is not an option, I need some way to tell my tests to look in ../ for interpreter sources. I was hoping there was a common idiom less cringeworthy than sys.path=['..']+sys.path. It seems like this would be a common problem, but maybe I'm just over-thinking it.
Cheers, fijal
Thanks, Terrence
On Wed, Aug 12, 2009 at 6:44 PM, Terrence Cole<list-sink@trainedmonkeystudios.org> wrote:
On Tue, 2009-08-11 at 03:16 -0600, Maciej Fijalkowski wrote:
Hi.
It's really cool that you find pypy as a good platform for writing interpreters, we're definitely happy with that :-) In general, we try to remove stuff from lang rather then put more of it there. So js/smalltalk/befunge interpreters would hopefully soon become projects on its own.
Thank you for the reply. I was thinking that the current situation in the lang directory would be unmanagable and I was wondering what I was missing :-).
I have not run into any documentation on making the pypy toolchain run from an installed location, rather than from the svn checkout dir. Since I don't even understand how autopath works yet, I figured that getting something more sophisticated working than what everyone else is doing in the lang directory would be an adventure better left for later. Is there a plan for making pypy installable? Is it already possible and I just haven't looked hard enough?
We should make it installable by say easy_install at some point, but for now you either need to set PYTHONPATH env variable, or simply add something like: import sys sys.path.insert(0, <path to pypy>) on top of your script. Cheers, fijal
participants (3)
-
Ben Mellor -
Maciej Fijalkowski -
Terrence Cole