Pie-thon benchmark code ready
While it's still 2003 in most of the US and Europe (and happy new year to the folks in Asia, Australia and New Zealand! :-), I present the official Pie-thon Parrot Benchmark: ftp://ftp.python.org/pub/python/parrotbench/parrotbench.tgz I'll quote from the README.txt file: """ This is a benchmark to be run in front of a live audience at OSCON 2004 between Python and Parrot. The bytecode must be Python 2.3 bytecode frozen in December 2003 (which is almost over as I write this :-). For some more background, see the python-dev thread around http://mail.python.org/pipermail/python-dev/2003-December/040977.html The benchmark here is intended to make Dan Sugalski's life difficult: there are some standard benchmark thingies (simple random algorithms using basic data types) but also a lot of play with odd corners of the language definition, and Python's extremely dynamic object model: funky descriptors, mutable classes, that sort of thing. The benchmark is supposed to run with either Python 2.3 or Python 2.4. [...] On timing: there's a requirement that the benchmark runs for at least 30 seconds. It currently runs for nearly a minute on my home box, which is a four-year-old 650 MHz Pentium box. If the contest hardware is so fast that it runs under a minute, there's a number in b.py that can be cranked up to increase the number of runs. (It takes 27 seconds on my recent work desktop, and on my screaming IBM T40 laptop it runs in 15 seconds, so I suspect that we should at least double the number of runs from 2 to 4.) """ I'd be happy to answer any questions. Let the competition begin! --Guido van Rossum (home page: http://www.python.org/~guido/)
At 9:54 AM -0800 12/31/03, Guido van Rossum wrote:
While it's still 2003 in most of the US and Europe (and happy new year to the folks in Asia, Australia and New Zealand! :-), I present the official Pie-thon Parrot Benchmark:
ftp://ftp.python.org/pub/python/parrotbench/parrotbench.tgz
Woohoo! The announcement's forwarded on to p6i for the denizens there to grovel over. I fully expect many hours of pain and misery ahead to make this work. :) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk
ftp://ftp.python.org/pub/python/parrotbench/parrotbench.tgz
Woohoo! The announcement's forwarded on to p6i for the denizens there to grovel over. I fully expect many hours of pain and misery ahead to make this work. :)
Enjoy. :-) Is there a online archive where I can watch the fun without signing up? BTW I forgot to mention that parts of the test are self-checking, but other parts require comparing of the output. The file 'out' is all you should care about. And if you want the bytecode, try this in a Unix shell: $ python2.3 -O [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import b ^D $
You should now have optimized bytecode in the 8 files b*.pyo. --Guido van Rossum (home page: http://www.python.org/~guido/)
At 10:27 AM -0800 12/31/03, Guido van Rossum wrote:
ftp://ftp.python.org/pub/python/parrotbench/parrotbench.tgz
Woohoo! The announcement's forwarded on to p6i for the denizens there to grovel over. I fully expect many hours of pain and misery ahead to make this work. :)
Enjoy. :-) Is there a online archive where I can watch the fun without signing up?
Yep. There's a news<->mail gateway for all the perl.org lists with web archive, so you can read via a newsreader at news://nntp.perl.org/perl.perl6.internals or webbed at http://www.nntp.perl.org/group/perl.perl6.internals. It's a bit of a rough web archive, unfortunately, but the gateway does maintain threads. (Assuming the participants aren't using a broken mail program. Alas, I sometimes do) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk
At 10:27 AM -0800 12/31/03, Guido van Rossum wrote:
ftp://ftp.python.org/pub/python/parrotbench/parrotbench.tgz
Woohoo! The announcement's forwarded on to p6i for the denizens there to grovel over. I fully expect many hours of pain and misery ahead to make this work. :)
Enjoy. :-) Is there a online archive where I can watch the fun without signing up?
BTW I forgot to mention that parts of the test are self-checking, but other parts require comparing of the output. The file 'out' is all you should care about. And if you want the bytecode, try this in a Unix shell:
$ python2.3 -O [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import b ^D $
Hrm. On my OS X laptop: lir:~/Desktop/parrotbench dan$ python -O Python 2.3 (#1, Sep 13 2003, 00:49:11) [GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin Type "help", "copyright", "credits" or "license" for more information.
import b --> iteration 0 --> b0 Traceback (most recent call last): File "<stdin>", line 1, in ? File "b.py", line 12, in ? b0.main() File "b0.py", line 893, in main checkoutput(4201300315) File "b0.py", line 763, in checkoutput check(strhash(outputtext), n) File "b0.py", line 3, in check raise AssertionError("%.30r != %.30r" % (a, b)) AssertionError: 503203581L != 4201300315L
I'll try generating the bytecode on a linux box, but something seems amiss somewhere. (Perhaps just in my understanding of what should be happening) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk
(Thanks for the p6i archives link BTW; Googling for "p6i" wasn't too helpful since its true name is perl6.internals. :-)
$ python2.3 -O [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import b ^D $
Hrm. On my OS X laptop:
lir:~/Desktop/parrotbench dan$ python -O Python 2.3 (#1, Sep 13 2003, 00:49:11) [GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin Type "help", "copyright", "credits" or "license" for more information.
import b --> iteration 0 --> b0 Traceback (most recent call last): File "<stdin>", line 1, in ? File "b.py", line 12, in ? b0.main() File "b0.py", line 893, in main checkoutput(4201300315) File "b0.py", line 763, in checkoutput check(strhash(outputtext), n) File "b0.py", line 3, in check raise AssertionError("%.30r != %.30r" % (a, b)) AssertionError: 503203581L != 4201300315L
I'll try generating the bytecode on a linux box, but something seems amiss somewhere. (Perhaps just in my understanding of what should be happening)
No, this means that the benchmark fails on that platform. But you got quite far (the first 5 sections of b0.py worked without a hitch). I notice you're using the original Python 2.3 release thjat came with Panther. It's a bit old. If upgrading to Python 2.3.3 doesn't help, can you send me the output of python b0.py ? Send it in private mail, it's rather large. :-) --Guido van Rossum (home page: http://www.python.org/~guido/)
At 11:31 AM -0800 12/31/03, Guido van Rossum wrote:
(Thanks for the p6i archives link BTW; Googling for "p6i" wasn't too helpful since its true name is perl6.internals. :-)
$ python2.3 -O [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
import b ^D $
Hrm. On my OS X laptop:
[Failure snipped]
I'll try generating the bytecode on a linux box, but something seems amiss somewhere. (Perhaps just in my understanding of what should be happening)
No, this means that the benchmark fails on that platform.
But you got quite far (the first 5 sections of b0.py worked without a hitch).
I notice you're using the original Python 2.3 release thjat came with Panther. It's a bit old. If upgrading to Python 2.3.3 doesn't help,
Turns out not to. I built and installed the 2.3.3 kit and it still fails, albeit differently.
can you send me the output of
python b0.py
? Send it in private mail, it's rather large. :-)
Sent under a separate cover. Traceback (for those folks following along at home) is: lir:~/Desktop/parrotbench dan$ /usr/local/bin/python2.3 b0.py >b0_output_darwin.lst Traceback (most recent call last): File "b0.py", line 924, in ? main() File "b0.py", line 893, in main checkoutput(4201300315) File "b0.py", line 763, in checkoutput check(strhash(outputtext), n) File "b0.py", line 3, in check raise AssertionError("%.30r != %.30r" % (a, b)) AssertionError: 2772929724L != 4201300315L lir:~/Desktop/parrotbench dan$ I did get bytecode on a linux system, so I've something to work with, though it'll be tough to test times as we go on. (Though I figure if I can beat the 1.4GHz Celeron system with my 600MHz G3 laptop I'll be doing good :) I'll give this a shot on the 10.2 OS X server box I have at home, just for chuckles. (I can set it up as an automated build server if you want, which is all I use it for right now) -- Dan --------------------------------------"it's like this"------------------- Dan Sugalski even samurai dan@sidhe.org have teddy bears and even teddy bears get drunk
>> > Hrm. On my OS X laptop: Dan> [Failure snipped] >> I notice you're using the original Python 2.3 release thjat came with >> Panther. It's a bit old. If upgrading to Python 2.3.3 doesn't help, Dan> Turns out not to. I built and installed the 2.3.3 kit and it still Dan> fails, albeit differently. Works for me (Mac OS X 10.2.8, Python 2.3.2, GCC 3.3, parrotbench directory after cvs up around 5:30PM central time): montanaro:parrotbench% python2.3 -O Python 2.3.2 (#1, Dec 17 2003, 17:22:01) [GCC 3.3 20030304 (Apple Computer, Inc. build 1493)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import b --> iteration 0 --> b0 3141592653 3141592653 --> b1 False False 0 --> b2 ... --> b4 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] --> b5 OK. --> b6 --> All done. Did you make sure you didn't have any .py[co] files before starting? Skip
Dan> Turns out not to. I built and installed the 2.3.3 kit and it still Dan> fails, albeit differently.
Works for me (Mac OS X 10.2.8, Python 2.3.2, GCC 3.3, parrotbench directory after cvs up around 5:30PM central time):
Check the README.txt. Does it say 1.0.1? Then you have my fixed version! I hope so, then I can go home. :-)
montanaro:parrotbench% python2.3 -O Python 2.3.2 (#1, Dec 17 2003, 17:22:01) [GCC 3.3 20030304 (Apple Computer, Inc. build 1493)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import b --> iteration 0 --> b0 3141592653 3141592653 --> b1 False False
^^^^^ This actually means that you failed the stack overflow test. It should work when you run it like this: python -O b.py Or you could do sys.setrecursionlimit(1001) before importing b in the interactive session. But that's unrelated to Dan's problem (which happens before b0 completes). And even if that test fails, the .pyo files will be correct. --Guido van Rossum (home page: http://www.python.org/~guido/)
>> Works for me (Mac OS X 10.2.8, Python 2.3.2, GCC 3.3, parrotbench >> directory after cvs up around 5:30PM central time): Guido> Check the README.txt. Does it say 1.0.1? Then you have my fixed Guido> version! Yes. Guido> I hope so, then I can go home. :-) I saw your checkin after my cvs up, so yes, you probably did fix the problem. Time to go find some champagne. Skip
participants (3)
-
Dan Sugalski -
Guido van Rossum -
Skip Montanaro