[Speed] Buildbot Status

[Maciej Fijalkowski]

On Mon, Jan 30, 2012 at 7:56 PM, Brett Cannon <brett at python.org> wrote:
>
>
> On Thu, Jan 26, 2012 at 15:21, Carsten Senger <senger at rehfisch.de> wrote:
>>
>> Hi everybody
>>
>> With the help of Maciej I worked on the buildbot in the last days. It
>> can build cpython, run the benchmarks and upload the results to one or
>> more codespeed instances. Maciej will look at the changes so we will
>> hopefully have a working buildbot for python 2.7 in the next days.
>>
>> This has a ticket in pypy's bugtracker: https://bugs.pypy.org/issue1015
>>
>> I also have a script we can use to run the benchmarks for parts of the
>> history and get data for a year or so into codespeed. The question is if
>> this data is interesting to anyone.
>
>
> I would say "don't worry about it unless you have some personal motivation
> to want to bother". While trending data is interesting, it isn't critical
> and a year will eventually pass anyway. =)
>
>>
>>
>>
>> What are the plans for benchmarking python 3?
>> How much of the benchmark suite will work with python 3, or can be made
>> work without much effort? Porting the runner and the support code is
>> easy, but directly porting the benchmarks including the used libraries
>> seems unrealistic.
>>
>> Can we replace them with newer versions that support python3 to get some
>> benchmarks working? Or build a second set of python3 compatible
>> benchmarks with these newer versions?
>>
>
> That's an open question. Until the libraries the benchmarks get ported
> officially then it's up in the air when the pre-existing benchmarks can
> move. We might have to look at pulling in a new set to start and then add
> back in the old ones (possibly) as they get ported.

Changing benchmarks is *never* a good idea. Note that we have quite
some history of those benchmarks running on pypy and I would strongly
object changing them in any way. Adding python 3 versions next to them
is much better. Also porting runner etc. is not a very good idea I
think.

The problem really is that most of interesting benchmarks don't work
on python 3, only the uninteresting ones. What we gonna do about that?

PS. I pulled your changes