Mailman 3 July 2012 - Speed

Re: [Speed] merging PyPy and Python benchmark suite
by Maciej Fijalkowski 23 Jul '12

23 Jul '12

On Mon, Jul 23, 2012 at 11:46 PM, Brett Cannon <brett(a)python.org> wrote: > > > On Mon, Jul 23, 2012 at 4:39 PM, Armin Rigo <arigo(a)tunes.org> wrote: >> >> Hi Brett, >> >> On Mon, Jul 23, 2012 at 10:15 PM, Brett Cannon <brett(a)python.org> wrote: >> > That's what I'm trying to establish; how much have they diverged and if >> > I'm >> > looking in the proper place. >> >> bm_mako.py is not from Unladen Swallow; that's why it is in >> pypy/benchmarks/own/. In case of doubts, check it in the history of >> Hg. The PyPy version was added from virhilo, which seems to be the >> name of his author, on 2010-12-21, and was not changed at all since >> then. > > > OK. Maciej has always told me that a problem with the Unladen benchmarks was > that some of them had artificial loop unrolling, etc., so I had assumed you > had simply fixed those instances instead of creating entirely new > benchmarks. No we did not use those benchmarks. Those were mostly completely artificial microbenchmarks (call, call_method etc.). We decided we're not really that interested in microbenchmarks. > >> >> >> Hg tells me that there was no change at all in the 'unladen_swallow' >> subdirectory, apart from 'unladen_swallow/perf.py' and adding some >> __init__.py somewhere. So at least these benchmarks did not receive >> any pypy-specific adapatations. If there are divergences, they come >> from changes done to the unladen-swallow benchmark suite after PyPy >> copied it on 2010-01-15. > > > I know that directory wasn't changed, but I also noticed that some > benchmarks had the same name, which is why I thought they were forked > versions of the same-named Unladen benchmarks. Not if they're in own/ directory. Cheers, fijal

1 0

Re: [Speed] merging PyPy and Python benchmark suite
by Alex Gaynor 23 Jul '12

23 Jul '12

On Mon, Jul 23, 2012 at 2:46 PM, Brett Cannon <brett(a)python.org> wrote: > > > On Mon, Jul 23, 2012 at 4:39 PM, Armin Rigo <arigo(a)tunes.org> wrote: > >> Hi Brett, >> >> On Mon, Jul 23, 2012 at 10:15 PM, Brett Cannon <brett(a)python.org> wrote: >> > That's what I'm trying to establish; how much have they diverged and if >> I'm >> > looking in the proper place. >> >> bm_mako.py is not from Unladen Swallow; that's why it is in >> pypy/benchmarks/own/. In case of doubts, check it in the history of >> Hg. The PyPy version was added from virhilo, which seems to be the >> name of his author, on 2010-12-21, and was not changed at all since >> then. >> > > OK. Maciej has always told me that a problem with the Unladen benchmarks > was that some of them had artificial loop unrolling, etc., so I had assumed > you had simply fixed those instances instead of creating entirely new > benchmarks. > > That was only the super-micro benchmarks like call_bench. We simply did not include them. > >> Hg tells me that there was no change at all in the 'unladen_swallow' >> subdirectory, apart from 'unladen_swallow/perf.py' and adding some >> __init__.py somewhere. So at least these benchmarks did not receive >> any pypy-specific adapatations. If there are divergences, they come >> from changes done to the unladen-swallow benchmark suite after PyPy >> copied it on 2010-01-15. >> > > I know that directory wasn't changed, but I also noticed that some > benchmarks had the same name, which is why I thought they were forked > versions of the same-named Unladen benchmarks. > > -Brett > > >> >> >> A bientôt, >> >> Armin. >> > > > _______________________________________________ > Speed mailing list > Speed(a)python.org > http://mail.python.org/mailman/listinfo/speed > > Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero

1 0

Re: [Speed] merging PyPy and Python benchmark suite
by Brett Cannon 23 Jul '12

23 Jul '12

On Mon, Jul 23, 2012 at 4:39 PM, Armin Rigo <arigo(a)tunes.org> wrote: > Hi Brett, > > On Mon, Jul 23, 2012 at 10:15 PM, Brett Cannon <brett(a)python.org> wrote: > > That's what I'm trying to establish; how much have they diverged and if > I'm > > looking in the proper place. > > bm_mako.py is not from Unladen Swallow; that's why it is in > pypy/benchmarks/own/. In case of doubts, check it in the history of > Hg. The PyPy version was added from virhilo, which seems to be the > name of his author, on 2010-12-21, and was not changed at all since > then. > OK. Maciej has always told me that a problem with the Unladen benchmarks was that some of them had artificial loop unrolling, etc., so I had assumed you had simply fixed those instances instead of creating entirely new benchmarks. > > Hg tells me that there was no change at all in the 'unladen_swallow' > subdirectory, apart from 'unladen_swallow/perf.py' and adding some > __init__.py somewhere. So at least these benchmarks did not receive > any pypy-specific adapatations. If there are divergences, they come > from changes done to the unladen-swallow benchmark suite after PyPy > copied it on 2010-01-15. > I know that directory wasn't changed, but I also noticed that some benchmarks had the same name, which is why I thought they were forked versions of the same-named Unladen benchmarks. -Brett > > > A bientôt, > > Armin. >

1 0

Re: [Speed] pathlib benchmark?
by Antoine Pitrou 23 Jul '12

23 Jul '12

On Mon, 23 Jul 2012 15:50:31 -0400 Brett Cannon <brett(a)python.org> wrote: > > Same here, especially if it can include Unicode path stuff. Ok, done in http://hg.python.org/benchmarks/rev/72516165f6ed. Unicode paths are naturally tested with 3.x, but not with 2.x where bytes are the natural type for file paths. (however, I didn't integrate any non-ASCII paths in the test, if that's what you meant) Regards Antoine. -- Software development and contracting: http://pro.pitrou.net

1 0

Re: [Speed] merging PyPy and Python benchmark suite
by Armin Rigo 23 Jul '12

23 Jul '12

Hi Brett, On Mon, Jul 23, 2012 at 10:15 PM, Brett Cannon <brett(a)python.org> wrote: > That's what I'm trying to establish; how much have they diverged and if I'm > looking in the proper place. bm_mako.py is not from Unladen Swallow; that's why it is in pypy/benchmarks/own/. In case of doubts, check it in the history of Hg. The PyPy version was added from virhilo, which seems to be the name of his author, on 2010-12-21, and was not changed at all since then. Hg tells me that there was no change at all in the 'unladen_swallow' subdirectory, apart from 'unladen_swallow/perf.py' and adding some __init__.py somewhere. So at least these benchmarks did not receive any pypy-specific adapatations. If there are divergences, they come from changes done to the unladen-swallow benchmark suite after PyPy copied it on 2010-01-15. A bientôt, Armin.

1 0

Re: [Speed] merging PyPy and Python benchmark suite
by Stefan Behnel 23 Jul '12

23 Jul '12

Brett Cannon, 23.07.2012 22:15: > On Mon, Jul 23, 2012 at 4:09 PM, Stefan Behnel wrote: >> The specific tests have >> certainly received some (minor) adaptations on both sides, be it for >> porting them to Py3 or for making them more appropriate for PyPy. > > That's what I'm trying to establish; how much have they diverged and if I'm > looking in the proper place. You should expect that the benchmarked libraries and applications are unchanged, and that only the (similarly named) benchmark scripts were modified. Stefan

1 0

Re: [Speed] merging PyPy and Python benchmark suite
by Stefan Behnel 23 Jul '12

23 Jul '12

Brett Cannon, 23.07.2012 21:48: > On Mon, Jul 23, 2012 at 2:33 PM, Stefan Behnel wrote: >> Brett Cannon, 23.07.2012 19:59: >>> For my keynotes at PyCon Argentina and Brasil I'm going to be talking >>> about Python 3.3 and trying to sell it to Python 2.7 users. That means >>> selling the performance numbers of Python 3.3 as acceptable for Python >>> 2.7 users. Since that means using the benchmark suite I might as well >>> start work on merging https://bitbucket.org/pypy/benchmarks/ and >>> http://hg.python.org/benchmarks so we can host the canonical >>> benchmarks at hg.python.org like we have discussed previously (along >>> with coming up with a way to run the benchmarks against Python 2.7 and >>> 3.3 for comparison and making more tests work in Python 3). >>> >>> So, to start, I want to remove the divergence of the Unladen >>> benchmarks (after that we can look at adding in any tests that are >>> unique to PyPy). Am I right in thinking that all forked tests that >>> came from Unladen ended up in PyPy's own directory under the same name >>> (e.g. bm_mako.py)? >> >> AFAICT, the only difference is the "ai" benchmark, which is otherwise known >> as "nqueens". > > That can't be true as bm_mako.py itself is different: compare > http://hg.python.org/benchmarks/file/5f6b46d86b40/performance/bm_mako.py > to https://bitbucket.org/pypy/benchmarks/src/ff7b35837d0f/own/bm_mako.py I meant "different" as in "differently named". The specific tests have certainly received some (minor) adaptations on both sides, be it for porting them to Py3 or for making them more appropriate for PyPy. Stefan

2 1

Re: [Speed] pathlib benchmark?
by Brett Cannon 23 Jul '12

23 Jul '12

On Mon, Jul 23, 2012 at 3:47 PM, Alex Gaynor <alex.gaynor(a)gmail.com> wrote: > > > On Mon, Jul 23, 2012 at 12:10 PM, Antoine Pitrou <solipsis(a)pitrou.net> > wrote: >> >> >> Hello, >> >> I have a small library named pathlib (*) and I think it may be a good >> candidate for a small benchmark since: >> - it doesn't have any dependencies >> - it doesn't need installing (the module is a single .py file) >> - it supports Python 2.7 and Python 3.2+ in the same code base >> >> I was thinking of a simple benchmark such as listing the entries of a >> given directory (prepared by the benchmark), or perhaps globbing for a >> specific file pattern. >> >> What do you think? >> >> (*) http://pathlib.readthedocs.org/en/latest/ >> >> Regards >> >> Antoine. >> >> >> -- >> Software development and contracting: http://pro.pitrou.net >> >> >> _______________________________________________ >> Speed mailing list >> Speed(a)python.org >> http://mail.python.org/mailman/listinfo/speed > > > Sounds like a reasonable idea to me. Same here, especially if it can include Unicode path stuff.

1 0

Re: [Speed] merging PyPy and Python benchmark suite
by Stefan Behnel 23 Jul '12

23 Jul '12

Brett Cannon, 23.07.2012 19:59: > For my keynotes at PyCon Argentina and Brasil I'm going to be talking > about Python 3.3 and trying to sell it to Python 2.7 users. That means > selling the performance numbers of Python 3.3 as acceptable for Python > 2.7 users. Since that means using the benchmark suite I might as well > start work on merging https://bitbucket.org/pypy/benchmarks/ and > http://hg.python.org/benchmarks so we can host the canonical > benchmarks at hg.python.org like we have discussed previously (along > with coming up with a way to run the benchmarks against Python 2.7 and > 3.3 for comparison and making more tests work in Python 3). > > So, to start, I want to remove the divergence of the Unladen > benchmarks (after that we can look at adding in any tests that are > unique to PyPy). Am I right in thinking that all forked tests that > came from Unladen ended up in PyPy's own directory under the same name > (e.g. bm_mako.py)? AFAICT, the only difference is the "ai" benchmark, which is otherwise known as "nqueens". Stefan

2 1

pathlib benchmark?
by Antoine Pitrou 23 Jul '12

23 Jul '12

Hello, I have a small library named pathlib (*) and I think it may be a good candidate for a small benchmark since: - it doesn't have any dependencies - it doesn't need installing (the module is a single .py file) - it supports Python 2.7 and Python 3.2+ in the same code base I was thinking of a simple benchmark such as listing the entries of a given directory (prepared by the benchmark), or perhaps globbing for a specific file pattern. What do you think? (*) http://pathlib.readthedocs.org/en/latest/ Regards Antoine. -- Software development and contracting: http://pro.pitrou.net

2 1