[pypy-dev] PyPy at last infinitely fast

[Miquel Torres]

Hi Antonio,

I didn't answer before because I was on holidays.

> I can see that the function finds the last_results to compare with simply by
> doing last_revs[1].  What about changing this logic to pick the latest rev
> which actually contains at least one result for the current environment?

That would be a quick workaround, yes, but we would quickly get into
trouble. How would you then know which revision to use to compute the
trend? A naive approach would be:
trend_rev = last_rev_with_data - 10 * number_of_revs_without_data (in
the current case 1 for 32 bit, 1 for 64 bit, which means 1)

Another would be
trend_rev = last_rev_with_data - 10 - number_of_revs_until_data_is_found

Those two revisions may or may not be the same.

So if there is no guarantee that every revision saved has data for
every executable in the project, there will be a random number of
executable groups (grouped by revision that is benchmarked). It *is*
solvable, but it can quickly become a mess. And we may want to show
other data or statistics in the future other than the trend.

That is the reason I suggest using different projects when different
executables will be using different revisions to do the benchmarking.
I can be proven wrong, however, and the workaround can be implemented
for changes, trend, and rev list without a performance penalty.

> uhm, I don't see any option to select a different project from within
> speed.pypy.org.  Is it simply because there is only one? Or having another
> project would mean to visit a complete different webpage?

First, speed.pypy.org only has a single project: PyPy (cpython exes
are under a CPython project, but they are not shown in the changes or
timeline view).
Second, even with several projects you will currently only see a list
of executables. Improving that is a theme for the GSoC

I will be able to work on this next week, so I maybe we can meet on
IRC and have a go at it? (and close this huge email thread ;-)


2011/7/28 Antonio Cuni <anto.cuni at gmail.com>:
> Hi Miquel,
>
> On 28/07/11 11:04, Miquel Torres wrote:
>>
>> Hi Antonio,
>>
>> I'm afraid you are right, the solution I proposed makes no sense.
>> Sorry I gave you a wrong answer. A revision is unique for a project
>> (well, now to a branch of a project), and thus they are not separated
>> by environment. Codespeed was not really designed with revisions in
>> mind that sometimes have results, and sometimes not. To solve that,
>> revisions would need to depend on an executable as well, or introduce
>> a check that so that the revision list is tailored to a particular
>> exe, but it would be ugly.
>
> I can think of a semi-ugly workaround, I don't know how close it is to a
> working solution.
>
> If I understand correctly, the numbers which are displayed in the "changes"
> page are precomputed and saved into "Reports" by
> create_report_if_enough_data.
>
> I can see that the function finds the last_results to compare with simply by
> doing last_revs[1].  What about changing this logic to pick the latest rev
> which actually contains at least one result for the current environment?
>
>> There is a way though to "solve" it right now. Separate executables
>> into two different projects. pypy32, pypy64, instead of different
>> environements. The revision list shown does change on-the-fly
>> depending on the project the selected exe belongs to.
>
> uhm, I don't see any option to select a different project from within
> speed.pypy.org.  Is it simply because there is only one? Or having another
> project would mean to visit a complete different webpage?
>
> Moreover, I wonder how this problem relates to the upcoming
> speed.python.org: will every interpreter (cpython, pypy, jython, etc.) be a
> separate project? Will it be possible to compare results of different
> interpreters?
>
> ciao,
> Anto
>