<div dir="ltr"><div class="gmail_default" style="color:rgb(0,0,0)"><br></div><div class="gmail_extra"><div class="gmail_quote">On Tue, Dec 1, 2015 at 6:36 AM, Maciej Fijalkowski <span dir="ltr"><<a href="mailto:fijall@gmail.com" target="_blank">fijall@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">Hi<br>
<br>
Thanks for doing the work! I'm on of the pypy devs and I'm very<br>
interested in seeing this getting somewhere. I must say I struggle to<br>
read the graph - is red good or is red bad for example?<br>
<br>
I'm keen to help you getting anything you want to run it repeatedly.<br>
<br>
PS. The intel stuff runs one benchmark in a very questionable manner,<br>
so let's maybe not rely on it too much.<br></blockquote><div><br></div><div><div class="gmail_default" style="color:rgb(0,0,0)">Hi Maciej,</div><div class="gmail_default" style="color:rgb(0,0,0)"><br></div><div class="gmail_default" style="color:rgb(0,0,0)">Great, it'd be awesome having data on multiple Python VMs (my latest target is really having a way to compare across multiple VMs/versions easily and help each implementation keep a focus on performance). Ideally, a single, dedicated machine could be used just to run the benchmarks from multiple VMs (one less variable to take into account for comparisons later on, as I'm not sure it'd be reliable to normalize benchmark data from different machines -- it seems Zach was the one to contact from that, but if there's such a machine already being used to run PyPy, maybe it could be extended to run other VMs too?).</div><div class="gmail_default" style="color:rgb(0,0,0)"><br></div><div class="gmail_default" style="color:rgb(0,0,0)">As for the graph, it should be easy to customize (and I'm open to suggestions). In the case, as it is, red is slower and blue is faster (so, for instance in <a href="https://www.speedtin.com/reports/1_CPython27x_Performance_Over_Time/" target="_blank" style="font-size:12.8px">https://www.speedtin.com/reports/1_CPython27x_Performance_Over_Time</a>, the fastest CPython version overall was 2.7.3 -- and 2.7.1 was the baseline). I've updated the comments to make it clearer (and changed the second graph to compare the latest against the fastest version (2.7.rc11 vs 2.7.3) for the individual benchmarks.</div><div class="gmail_default" style="color:rgb(0,0,0)"><br></div><div class="gmail_default" style="color:rgb(0,0,0)">Best Regards,</div><div class="gmail_default" style="color:rgb(0,0,0)"><br></div><div class="gmail_default" style="color:rgb(0,0,0)">Fabio</div><br></div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<span class=""><br>
On Mon, Nov 30, 2015 at 3:52 PM, R. David Murray <<a href="mailto:rdmurray@bitdance.com">rdmurray@bitdance.com</a>> wrote:<br>
> On Mon, 30 Nov 2015 09:02:12 -0200, Fabio Zadrozny <<a href="mailto:fabiofz@gmail.com">fabiofz@gmail.com</a>> wrote:<br>
>> Note that uploading the data to SpeedTin should be pretty straightforward<br>
>> (by using <a href="https://github.com/fabioz/pyspeedtin" rel="noreferrer" target="_blank">https://github.com/fabioz/pyspeedtin</a>, so, the main issue would be<br>
>> setting up o machine to run the benchmarks).<br>
><br>
> Thanks, but Zach almost has this working using codespeed (he's still<br>
> waiting on a review from infrastructure, I think). The server was not in<br>
> fact running; a large part of what Zach did was to get that server set up.<br>
> I don't know what it would take to export the data to another consumer,<br>
> but if you want to work on that I'm guessing there would be no objection.<br>
> And I'm sure there would be no objection if you want to get involved<br>
> in maintaining the benchmark server!<br>
><br>
> There's also an Intel project posted about here recently that checks<br>
> individual benchmarks for performance regressions and posts the results<br>
> to python-checkins.<br>
><br>
> --David<br>
> _______________________________________________<br>
> Python-Dev mailing list<br>
> <a href="mailto:Python-Dev@python.org">Python-Dev@python.org</a><br>
> <a href="https://mail.python.org/mailman/listinfo/python-dev" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/python-dev</a><br>
</span>> Unsubscribe: <a href="https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com" rel="noreferrer" target="_blank">https://mail.python.org/mailman/options/python-dev/fijall%40gmail.com</a><br>
<div class=""><div class="h5">_______________________________________________<br>
Python-Dev mailing list<br>
<a href="mailto:Python-Dev@python.org">Python-Dev@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/python-dev" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/python-dev</a><br>
Unsubscribe: <a href="https://mail.python.org/mailman/options/python-dev/fabiofz%40gmail.com" rel="noreferrer" target="_blank">https://mail.python.org/mailman/options/python-dev/fabiofz%40gmail.com</a><br>
</div></div></blockquote></div><br></div></div>