Hi Miquel, all, regarding the new speed.pypy.org host I realized that the current setting is not suited for setting up another VM. Too few resources and old-ish software. I plan to order a new host, setup speed.pypy.org there (and migrate existing VMs later). Takes a week or two i guess. Hope this doesn't block you. Could maybe someone here on the list (temporarily) offer a suitable VM to run Django Apps/DBs where Miquel can get an account? i'd then point the "speed.pypy.org" DNS entry to it. The earlier the site gets running the earlier we can all follow speed developments with the PyPy JIT :) best, holger -- Metaprogramming, Python, Testing: http://tetamap.wordpress.com Python, PyPy, pytest contracting: http://merlinux.eu
Hi Holger, for the time being it is not a problem. I can begin the implementation locally and later continue on the server VM. One thing I would need though: a speed.pypy wiki page. I would like that you (pypy core developers) write down the most important use cases for speed.pypy.org For example: - See right away the last x trunk revisions. A line plot (and table?) will be shown for each individual benchmark in the benchmark suite. - Allow to select multiple, arbitrary trunk revisions for comparing - Show different data series in the same graph for different compile options (like tuatara's benchmarks??) ... I have my own ideas, but you will be the main users, so you need to tell me what you need implemented first. Another wiki page could be about what benchmarks could be useful for the suite. It would be great if all benchmarks results were points, or all seconds (less is better), (or x times faster/slower than cpython) because that way results can be shown in a much more compact form. For example stacked: http://www.codinghorror.com/blog/images/browser-javascript-performance-graph... * * or combined in a single chart: http://www.codinghorror.com/blog/images/browser-javascript-performance-graph... Anyway the lack of a VM is not a problem for me right now. Cheers, Miquel <http://www.codinghorror.com/blog/images/browser-javascript-performance-graph-breakdown-1.png> 2009/10/2 holger krekel <holger@merlinux.eu>
Hi Miquel, all,
regarding the new speed.pypy.org host I realized that the current setting is not suited for setting up another VM. Too few resources and old-ish software. I plan to order a new host, setup speed.pypy.org there (and migrate existing VMs later). Takes a week or two i guess. Hope this doesn't block you.
Could maybe someone here on the list (temporarily) offer a suitable VM to run Django Apps/DBs where Miquel can get an account? i'd then point the "speed.pypy.org" DNS entry to it. The earlier the site gets running the earlier we can all follow speed developments with the PyPy JIT :)
best, holger
-- Metaprogramming, Python, Testing: http://tetamap.wordpress.com Python, PyPy, pytest contracting: http://merlinux.eu
Hi Miquel, May I point out a couple of comments about http://speed.pypy.org/ ? The first is that it looks great, indeed; thank you very much for doing such a site! :-) The comments are mostly about the graphics in /timeline: * they should also display, or allow to display, the speed of CPython for comparison; * they should have a longer maximum history than 100, to see more clearly the long-term evolution. All in all it looks great! A bientot, Armin.
I also have a bunch of comments, all are my opinion and should not be taken as demands (or even as a good review). - When you first visit the site I think it would be better to be on the timeline like on http://buildbot.pypy.org/plotsummary.html that shows all benchmarks on the same page, but just the last 50 revisions or it becomes too hard to see the recent improvements. (being able to show more history is important too, maybe there is a way to show the trend and the last x revisions clearly). - Overview window comments: - rename the column "result" to "time" (or timing?); - create a new column called "previous time" with the last time of the previous revision. - rename "current change" to "improvement" and invert the values (instead of a green -2% you end up with a green 2%); - with the last two ones it will make understanding the improvement and trends a lot easier; - trend should have the number of revisions on the label (and the values reversed like the "improvement" column); - rename the column "times cpython" to something else (eg. "compared to cpython") and maybe have values like "2x slow down" and "2.5x speedup"; - the graph should say what it is graphing; - maybe invert the order of results. - Timeline window comments: - showing all benchmarks in the same page would be good - Pypy with jit should come first in the list of interpreters, and while you only test with hybrid gc there is no need to state that explicitly; - see all benchmarks together, so you don't have to hunt around (maybe have a detailed view); - describe all axis; - as armin said have a selected by default cpython comparison so people can have a baseline. I think this speed center could be used for the common mercurial benchmark repository that is being setup, that would be very very cool. On Feb 25, 2010, at 6:18 AM, Armin Rigo wrote:
Hi Miquel,
May I point out a couple of comments about http://speed.pypy.org/ ? The first is that it looks great, indeed; thank you very much for doing such a site! :-)
The comments are mostly about the graphics in /timeline:
* they should also display, or allow to display, the speed of CPython for comparison;
* they should have a longer maximum history than 100, to see more clearly the long-term evolution.
All in all it looks great!
A bientot, Armin. _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
-- Leonardo Santagada santagada at gmail.com
Thanks for the nice comments. I was waiting for the test hook to be implemented before emailing pypy-dev about it, but seeing as It already got out I may do it now as well ;-) @Armin
* they should also display, or allow to display, the speed of CPython for comparison; Already planned.
* they should have a longer maximum history than 100, to see more clearly the long-term evolution. Done
@Leonardo phew! those were a lot of suggestions, thanks. I will start implementing some of them, though I would like the changes to be discussed so that they get done right and all agree on what's best (in the cases where it can't be an option). I already implemented a couple of changes suggested by fijal: logarithmic scale for the overview bars and variable width so it fits on small screens, down to 980px width. I hope he is happy now :-) I'll start a new thread for the explanation and plans. Cheers! 2010/2/25 Leonardo Santagada <santagada@gmail.com>:
I also have a bunch of comments, all are my opinion and should not be taken as demands (or even as a good review).
- When you first visit the site I think it would be better to be on the timeline like on http://buildbot.pypy.org/plotsummary.html that shows all benchmarks on the same page, but just the last 50 revisions or it becomes too hard to see the recent improvements. (being able to show more history is important too, maybe there is a way to show the trend and the last x revisions clearly).
- Overview window comments: - rename the column "result" to "time" (or timing?); - create a new column called "previous time" with the last time of the previous revision. - rename "current change" to "improvement" and invert the values (instead of a green -2% you end up with a green 2%); - with the last two ones it will make understanding the improvement and trends a lot easier; - trend should have the number of revisions on the label (and the values reversed like the "improvement" column); - rename the column "times cpython" to something else (eg. "compared to cpython") and maybe have values like "2x slow down" and "2.5x speedup"; - the graph should say what it is graphing; - maybe invert the order of results.
- Timeline window comments: - showing all benchmarks in the same page would be good - Pypy with jit should come first in the list of interpreters, and while you only test with hybrid gc there is no need to state that explicitly; - see all benchmarks together, so you don't have to hunt around (maybe have a detailed view); - describe all axis; - as armin said have a selected by default cpython comparison so people can have a baseline.
I think this speed center could be used for the common mercurial benchmark repository that is being setup, that would be very very cool.
On Feb 25, 2010, at 6:18 AM, Armin Rigo wrote:
Hi Miquel,
May I point out a couple of comments about http://speed.pypy.org/ ? The first is that it looks great, indeed; thank you very much for doing such a site! :-)
The comments are mostly about the graphics in /timeline:
* they should also display, or allow to display, the speed of CPython for comparison;
* they should have a longer maximum history than 100, to see more clearly the long-term evolution.
All in all it looks great!
A bientot, Armin. _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
-- Leonardo Santagada santagada at gmail.com
I have implemented some of Leonardo's suggestions:
- Overview window comments: - rename the column "result" to "time" (or timing?); done - trend should have the number of revisions on the label (and the values reversed like the "improvement" column); done - rename the column "times cpython" to something else (eg. "compared to cpython") and maybe have values like "2x slow down" and "2.5x speedup"; - the graph should say what it is graphing; it is clearer now, plus there are some tooltips on hover.
- Timeline window comments: - Pypy with jit should come first in the list of interpreters, and while you only test with hybrid gc there is no need to state that explicitly; only interpreter name shown now. (order implemented but not commited).
I'll see about the rest when I have time. Miquel 2010/2/25 Leonardo Santagada <santagada@gmail.com>:
I also have a bunch of comments, all are my opinion and should not be taken as demands (or even as a good review).
- When you first visit the site I think it would be better to be on the timeline like on http://buildbot.pypy.org/plotsummary.html that shows all benchmarks on the same page, but just the last 50 revisions or it becomes too hard to see the recent improvements. (being able to show more history is important too, maybe there is a way to show the trend and the last x revisions clearly).
- Overview window comments: - rename the column "result" to "time" (or timing?); - create a new column called "previous time" with the last time of the previous revision. - rename "current change" to "improvement" and invert the values (instead of a green -2% you end up with a green 2%); - with the last two ones it will make understanding the improvement and trends a lot easier; - trend should have the number of revisions on the label (and the values reversed like the "improvement" column); - rename the column "times cpython" to something else (eg. "compared to cpython") and maybe have values like "2x slow down" and "2.5x speedup"; - the graph should say what it is graphing; - maybe invert the order of results.
- Timeline window comments: - showing all benchmarks in the same page would be good - Pypy with jit should come first in the list of interpreters, and while you only test with hybrid gc there is no need to state that explicitly; - see all benchmarks together, so you don't have to hunt around (maybe have a detailed view); - describe all axis; - as armin said have a selected by default cpython comparison so people can have a baseline.
I think this speed center could be used for the common mercurial benchmark repository that is being setup, that would be very very cool.
On Feb 25, 2010, at 6:18 AM, Armin Rigo wrote:
Hi Miquel,
May I point out a couple of comments about http://speed.pypy.org/ ? The first is that it looks great, indeed; thank you very much for doing such a site! :-)
The comments are mostly about the graphics in /timeline:
* they should also display, or allow to display, the speed of CPython for comparison;
* they should have a longer maximum history than 100, to see more clearly the long-term evolution.
All in all it looks great!
A bientot, Armin. _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
-- Leonardo Santagada santagada at gmail.com
I have commited a bunch of changes: - Added cpython baseline to timeline - Changed timeline defaults to both interpreters selected (plus baseline) - Performance improvements in Timeline's ajax: due to above changes, I worried that for a lot of interpreters selected the queries could be too slow. Even though it is known that Premature optimization is the root of all evil (tm), I had a look and optimized django database calls a little bit (or more precisely made my implementation suck less). So instead of nearly 200 ms per interpreter, it now takes less than 100ms (measured locally, add >100ms for network latency). Plot rerendering should feel faster now. I find it interesting how quick django is. Sometimes you may even forget that the data is being pulled from the server each time a different plot is selected. - Timeline: Added labels for the axes - pypy-c-jit appears now listed first - Added tooltips for Host info and for some benchmarks. Copied the descriptions for the benchmarks listed at the unladen site. Cheers, Miquel 2010/2/25 Leonardo Santagada <santagada@gmail.com>:
I also have a bunch of comments, all are my opinion and should not be taken as demands (or even as a good review).
- When you first visit the site I think it would be better to be on the timeline like on http://buildbot.pypy.org/plotsummary.html that shows all benchmarks on the same page, but just the last 50 revisions or it becomes too hard to see the recent improvements. (being able to show more history is important too, maybe there is a way to show the trend and the last x revisions clearly).
- Overview window comments: - rename the column "result" to "time" (or timing?); - create a new column called "previous time" with the last time of the previous revision. - rename "current change" to "improvement" and invert the values (instead of a green -2% you end up with a green 2%); - with the last two ones it will make understanding the improvement and trends a lot easier; - trend should have the number of revisions on the label (and the values reversed like the "improvement" column); - rename the column "times cpython" to something else (eg. "compared to cpython") and maybe have values like "2x slow down" and "2.5x speedup"; - the graph should say what it is graphing; - maybe invert the order of results.
- Timeline window comments: - showing all benchmarks in the same page would be good - Pypy with jit should come first in the list of interpreters, and while you only test with hybrid gc there is no need to state that explicitly; - see all benchmarks together, so you don't have to hunt around (maybe have a detailed view); - describe all axis; - as armin said have a selected by default cpython comparison so people can have a baseline.
I think this speed center could be used for the common mercurial benchmark repository that is being setup, that would be very very cool.
On Feb 25, 2010, at 6:18 AM, Armin Rigo wrote:
Hi Miquel,
May I point out a couple of comments about http://speed.pypy.org/ ? The first is that it looks great, indeed; thank you very much for doing such a site! :-)
The comments are mostly about the graphics in /timeline:
* they should also display, or allow to display, the speed of CPython for comparison;
* they should have a longer maximum history than 100, to see more clearly the long-term evolution.
All in all it looks great!
A bientot, Armin. _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
-- Leonardo Santagada santagada at gmail.com
Hi there Armin Rigo wrote:
Hi Miquel,
May I point out a couple of comments about http://speed.pypy.org/ ? The first is that it looks great, indeed; thank you very much for doing such a site! :-)
Yes - it looks both nice and very useful indeed! Do we have a link from PyPys "main page" http://codespeak.net/pypy/dist/pypy/doc/ to this page so people can find it more easily? Or did I miss that? Cheers Bea
The comments are mostly about the graphics in /timeline:
* they should also display, or allow to display, the speed of CPython for comparison;
* they should have a longer maximum history than 100, to see more clearly the long-term evolution.
All in all it looks great!
A bientot, Armin. _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
On Fri, Feb 26, 2010 at 5:44 AM, Bea During <bea@changemaker.nu> wrote:> Hi there
Armin Rigo wrote:
Hi Miquel,
May I point out a couple of comments about http://speed.pypy.org/ ? The first is that it looks great, indeed; thank you very much for doing such a site! :-)
Yes - it looks both nice and very useful indeed!
Do we have a link from PyPys "main page" http://codespeak.net/pypy/dist/pypy/doc/ to this page so people can find it more easily?
Or did I miss that?
Our main website is a mess right now. I would like to have it redesigned, so we have actually interesting links from there, and obviously the one to graphs (we have two of those now) would be great. Note that for now speed.pypy.org is not automatically updated - it requires adjustments to buildbot run. Cheers, fijal
Am 26.02.10 16:05, schrieb Maciej Fijalkowski:
On Fri, Feb 26, 2010 at 5:44 AM, Bea During<bea@changemaker.nu> wrote:> Hi there
Armin Rigo wrote:
Hi Miquel,
May I point out a couple of comments about http://speed.pypy.org/ ? The first is that it looks great, indeed; thank you very much for doing such a site! :-)
Yes - it looks both nice and very useful indeed!
Do we have a link from PyPys "main page" http://codespeak.net/pypy/dist/pypy/doc/ to this page so people can find it more easily?
Or did I miss that?
Our main website is a mess right now. I would like to have it redesigned, so we have actually interesting links from there, and obviously the one to graphs (we have two of those now) would be great. Note that for now speed.pypy.org is not automatically updated - it requires adjustments to buildbot run.
Cheers, fijal _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev If there's the need I could to do a redesign of the layout and (if needed) structure. So you guys could focus on pypy development and just answer a few questions I come up with while working.
Greetings -- Samuel
On Fri, Feb 26, 2010 at 10:21 AM, Samuel Reis <con@samuelreis.de> wrote:
Am 26.02.10 16:05, schrieb Maciej Fijalkowski:
On Fri, Feb 26, 2010 at 5:44 AM, Bea During<bea@changemaker.nu> wrote:> Hi there
Armin Rigo wrote:
Hi Miquel,
May I point out a couple of comments about http://speed.pypy.org/ ? The first is that it looks great, indeed; thank you very much for doing such a site! :-)
Yes - it looks both nice and very useful indeed!
Do we have a link from PyPys "main page" http://codespeak.net/pypy/dist/pypy/doc/ to this page so people can find it more easily?
Or did I miss that?
Our main website is a mess right now. I would like to have it redesigned, so we have actually interesting links from there, and obviously the one to graphs (we have two of those now) would be great. Note that for now speed.pypy.org is not automatically updated - it requires adjustments to buildbot run.
Cheers, fijal _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev If there's the need I could to do a redesign of the layout and (if needed) structure. So you guys could focus on pypy development and just answer a few questions I come up with while working.
Great! I have tons of ideas and no actual time to do all this (and no skills as well). How about meeting on IRC at some point? It's easier to discuss stuff that on mail I'm fijal on #pypy at freenode.
Greetings -- Samuel _______________________________________________ pypy-dev@codespeak.net http://codespeak.net/mailman/listinfo/pypy-dev
participants (7)
-
Armin Rigo -
Bea During -
holger krekel -
Leonardo Santagada -
Maciej Fijalkowski -
Miquel Torres -
Samuel Reis