Re: [pypy-dev] Change to the frontpage of speed.pypy.org
In a message of Sun, 27 Feb 2011 19:36:52 +0100, Miquel Torres writes:
Hi Laura,
you bring up good points, however, it is not as straight forward as it se em=
Well, it really is a list of the latest results. The problem is that speed.pypy.org is foremost a tool to help in development. As such, the logic behind the "latest results" list is regression oriented, or let us say pessimistic.
For example, this revision: http://speed.pypy.org/changes/?rev=3D42312:392bbf936179
The average change is actually -0.91%, which is actually an improvement, though not an statistical significant one. However, There was a sizeable regression in spitfire_cstringio, +5.21. The "summary" for that revision is then "regression for an individual benchmark". Which is actually what developers need to know: they should check whether that revision really introduced a real regression in performance.
I understand this. I just don't think that this should be on our front page. Developers can get used to looking anywhere, and indeed I never use the front page for looking at anything important -- I am always looking at the complete stats for any runs. I think that speed.pypy.org -- the front page -- should primarily be of use to people who want to find out if pypy is good for them, people who want to convince their bosses that they should switch, and people who just want to cheer us on and add to the general warm feeling about pypy. I see it as the prime tool in the world domination project.
That said, I do understand where you are coming from. I would point outsiders though directly to http://speed.pypy.org/comparison/
This is the wrong way to do things from a usability point of view. What the casual person wants is not a way to dig down and get information, but something already packaged for them which already tells them the main story. Then they can dig down if they actually care. This is what Steve Krug calls the 'Don't Make Me Think' principle.
So what can some body think about what could be changed or added so that the main page doesn't give a negative impression to the non-developer?
Something I could think of is to add, above the results list, a plot showing the overall trend over the last 2 or 3 months. What do you think?
I think this would be a very nice thing to have as the home page for speed.pypy.org Then the page that we have now could be called something like http://speed.pypy.org/regressions or something.
The other thing I want is for the graphs you get, for instance with http://speed.pypy.org/changes/?rev=3D42312:392bbf936179&exe=3D%203&env= 3D= tannit to have, in addition to the selection button beside: 'result for revisi on= ' an actual label that says 'build 42312:392b' or something that you can select with your mouse and use to paste into things like this mail article.
I think to the right of the changes table there is a box with info for the revision, with a text field you can select and copy. Isn't that what you want?
Looking at http://speed.pypy.org/changes/?rev=42312:392bbf936179&exe=%203&env=tannit There is nothing to the right of the table. To the right of the label that says: Results for revision is a box that you can use to select different builds to look at. The text in this box is not selectable; you cannot paste it anywhere. To the right of that is a link that says 'Permalink'. As far as I can tell clicking it causes the page to refresh and nothing more. This is with iceweasel 3.5.16 (which is debian's repackage of firefox 3.5.16) Laura
Ok, I'll try that then.
There is nothing to the right of the table. What is your screen width? The Revision info box should be below the table if your width is small.
2011/2/27 Laura Creighton <lac@openend.se>:
In a message of Sun, 27 Feb 2011 19:36:52 +0100, Miquel Torres writes:
Hi Laura,
you bring up good points, however, it is not as straight forward as it se em=
Well, it really is a list of the latest results. The problem is that speed.pypy.org is foremost a tool to help in development. As such, the logic behind the "latest results" list is regression oriented, or let us say pessimistic.
For example, this revision: http://speed.pypy.org/changes/?rev=3D42312:392bbf936179
The average change is actually -0.91%, which is actually an improvement, though not an statistical significant one. However, There was a sizeable regression in spitfire_cstringio, +5.21. The "summary" for that revision is then "regression for an individual benchmark". Which is actually what developers need to know: they should check whether that revision really introduced a real regression in performance.
I understand this. I just don't think that this should be on our front page. Developers can get used to looking anywhere, and indeed I never use the front page for looking at anything important -- I am always looking at the complete stats for any runs.
I think that speed.pypy.org -- the front page -- should primarily be of use to people who want to find out if pypy is good for them, people who want to convince their bosses that they should switch, and people who just want to cheer us on and add to the general warm feeling about pypy. I see it as the prime tool in the world domination project.
That said, I do understand where you are coming from. I would point outsiders though directly to http://speed.pypy.org/comparison/
This is the wrong way to do things from a usability point of view. What the casual person wants is not a way to dig down and get information, but something already packaged for them which already tells them the main story. Then they can dig down if they actually care. This is what Steve Krug calls the 'Don't Make Me Think' principle.
So what can some body think about what could be changed or added so that the main page doesn't give a negative impression to the non-developer?
Something I could think of is to add, above the results list, a plot showing the overall trend over the last 2 or 3 months. What do you think?
I think this would be a very nice thing to have as the home page for speed.pypy.org Then the page that we have now could be called something like http://speed.pypy.org/regressions or something.
The other thing I want is for the graphs you get, for instance with http://speed.pypy.org/changes/?rev=3D42312:392bbf936179&exe=3D%203&env= 3D= tannit to have, in addition to the selection button beside: 'result for revisi on= ' an actual label that says 'build 42312:392b' or something that you can select with your mouse and use to paste into things like this mail article.
I think to the right of the changes table there is a box with info for the revision, with a text field you can select and copy. Isn't that what you want?
Looking at http://speed.pypy.org/changes/?rev=42312:392bbf936179&exe=%203&env=tannit
There is nothing to the right of the table. To the right of the label that says: Results for revision is a box that you can use to select different builds to look at. The text in this box is not selectable; you cannot paste it anywhere. To the right of that is a link that says 'Permalink'. As far as I can tell clicking it causes the page to refresh and nothing more.
This is with iceweasel 3.5.16 (which is debian's repackage of firefox 3.5.16)
Laura
Hey. Just my 5 cents. It would be cool if default view has a down-scaled version of comparison against CPython. I can look anywhere for recent changes. Also the recent changes as they're now are not very informative and I don't use them at all. They stick around, so I don't know if they're new or old. I'm also as interested in good as in bad changes. Simply this: http://speed.pypy.org/changes/ is way more informative. Can we either just remove the red recent changes for now or simply put a vs cpython, scaled down graph there? At least for pycon this seems like a better way to go. Cheers, fijal PS. Miquel, don't get me wrong, I think you're doing an awesome job, the speed website itself was a huge step forward for us.
Adding to Maciej's 5cents:
It would be cool if default view has a down-scaled version of comparison against CPython.
I for one second this suggestion. Comparison against cPython is the TLDR of speed.pypy.org; the abstract, the executive-level-information. Harald -- GHUM GmbH Harald Armin Massa Spielberger Straße 49 70435 Stuttgart 0173/9409607 Amtsgericht Stuttgart, HRB 734971 - persuadere. et programmare
participants (4)
-
Laura Creighton -
Maciej Fijalkowski -
Massa, Harald Armin -
Miquel Torres