<br><br><div class="gmail_quote">On Tue, Jul 24, 2012 at 10:10 AM, Brett Cannon <span dir="ltr"><<a href="mailto:brett@python.org" target="_blank">brett@python.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br><br><div class="gmail_quote"><div><div class="h5">On Mon, Jul 23, 2012 at 7:34 PM, Maciej Fijalkowski <span dir="ltr"><<a href="mailto:fijall@gmail.com" target="_blank">fijall@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>On Mon, Jul 23, 2012 at 11:46 PM, Brett Cannon <<a href="mailto:brett@python.org" target="_blank">brett@python.org</a>> wrote:<br>
><br>
><br>
> On Mon, Jul 23, 2012 at 4:39 PM, Armin Rigo <<a href="mailto:arigo@tunes.org" target="_blank">arigo@tunes.org</a>> wrote:<br>
>><br>
>> Hi Brett,<br>
>><br>
>> On Mon, Jul 23, 2012 at 10:15 PM, Brett Cannon <<a href="mailto:brett@python.org" target="_blank">brett@python.org</a>> wrote:<br>
>> > That's what I'm trying to establish; how much have they diverged and if<br>
>> > I'm<br>
>> > looking in the proper place.<br>
>><br>
>> bm_mako.py is not from Unladen Swallow; that's why it is in<br>
>> pypy/benchmarks/own/. In case of doubts, check it in the history of<br>
>> Hg. The PyPy version was added from virhilo, which seems to be the<br>
>> name of his author, on 2010-12-21, and was not changed at all since<br>
>> then.<br>
><br>
><br>
> OK. Maciej has always told me that a problem with the Unladen benchmarks was<br>
> that some of them had artificial loop unrolling, etc., so I had assumed you<br>
> had simply fixed those instances instead of creating entirely new<br>
> benchmarks.<br>
<br>
</div>No we did not use those benchmarks. Those were mostly completely<br>
artificial microbenchmarks (call, call_method etc.). We decided we're<br>
not really that interested in microbenchmarks.<br>
<div><br>
><br>
>><br>
>><br>
>> Hg tells me that there was no change at all in the 'unladen_swallow'<br>
>> subdirectory, apart from 'unladen_swallow/perf.py' and adding some<br>
>> __init__.py somewhere. So at least these benchmarks did not receive<br>
>> any pypy-specific adapatations. If there are divergences, they come<br>
>> from changes done to the unladen-swallow benchmark suite after PyPy<br>
>> copied it on 2010-01-15.<br>
><br>
><br>
> I know that directory wasn't changed, but I also noticed that some<br>
> benchmarks had the same name, which is why I thought they were forked<br>
> versions of the same-named Unladen benchmarks.<br>
<br>
</div>Not if they're in own/ directory.<br></blockquote><div><br></div></div></div><div>OK, good to know. I realized I can't copy code wholesale from PyPy's benchmark suite as I don't know the code's history and thus if the contributor signed Python's contributor agreement. Can the people who are familiar with the code help move benchmarks over where the copyright isn't in question?</div>
<div><br></div><div>I can at least try to improve the Python 3 situation by doing things like pulling in Vinay's py3k port of Django, etc. to fill in gaps. I will also try to get the benchmarks to work with a Python 2.7 control and a Python 3 "experimental" target for comparing performance since that's what I need (or at least be able to run the benchmarks on their own and writing out the results for later comparison).</div>
<div><br></div><div>Anything else that should be worked on?</div></div>
<br>_______________________________________________<br>
Speed mailing list<br>
<a href="mailto:Speed@python.org">Speed@python.org</a><br>
<a href="http://mail.python.org/mailman/listinfo/speed" target="_blank">http://mail.python.org/mailman/listinfo/speed</a><br>
<br></blockquote></div><br>The important thing is that once a benchmark is in the repo it can *never* change including all the versions of dependencies, only Python can vary, otherwise you kill the ability to actually do science with the numbers.<div>
<br></div><div>So, e.g., I wouldn't pull in Vijnay's fork, since that's going to be utterly obsolete in a few weeks probably, I'd wait to have django on py3k for that work to all be merged into django itself.</div>
<div><br></div><div>Alex<br clear="all"><div><br></div>-- <br>"I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)<br>"The people's good is the highest law." -- Cicero<br>
<br>
</div>