Speed September 2011

speed@python.org

8 participants
19 discussions

Re: [Speed] [pypy-dev] Moving the project forward
by Jesse Noller 08 Sep '11

08 Sep '11

Miquel, Noah and the people who know what needs to be ran should be able to fill you in. You're right to not want to duplicate or do busy work. On Sep 8, 2011, at 7:04 PM, Tennessee Leeuwenburg <tleeuwenburg(a)gmail.com> wrote: > I would be more than happy to be progressing the cpython benchmarking > automation, but I got no reply to my email summarising what needed to > be done based on the comments from the list... I'm worried that > anything I do will have to be undone if I don't follow the right path > in terms of setting up buildbots, using Chef, etc etc. > > Thanks, > -T > > On Fri, Sep 9, 2011 at 5:16 AM, Miquel Torres <tobami(a)googlemail.com> wrote: >> Hi Jesse, >> >> Sorry, I spent last week polishing up some things to be able to >> release Codespeed 0.9.1 and deploy that, because I had some Apache >> configuration issues... >> I'll have to try again this weekend. >> >> Cheers, >> Miquel >> >> >> 2011/9/8 Jesse Noller <jnoller(a)gmail.com>: >>> PING: Did we make progress? >>> >>> On Thu, Sep 1, 2011 at 2:44 PM, Miquel Torres <tobami(a)googlemail.com> wrote: >>>> You can also do that in Github, which I prefer. >>>> >>>> However, since CPython and PyPy use mercurial, the general preference >>>> for Bitbucket is understandable. >>>> >>>> >>>> 2011/9/1 Brett Cannon <brett(a)python.org>: >>>>> On Thu, Sep 1, 2011 at 01:10, Nick Coghlan <ncoghlan(a)gmail.com> wrote: >>>>>> On Thu, Sep 1, 2011 at 5:50 PM, Antonio Cuni <anto.cuni(a)gmail.com> wrote: >>>>>>> On 31/08/11 22:11, Brett Cannon wrote: >>>>>>>> >>>>>>>> The PyPy folk could answer this as they have their repo on bitbucket >>>>>>>> already. Else I guess we can just create a standalone account that >>>>>>>> represents the official speed.python.org account. >>>>>>> >>>>>>> for pypy we do exactly that. There is a bitbucket user named "pypy" whose >>>>>>> credentials are shared among all the core devs. >>>>>> >>>>>> The security auditing part of my brain has its fingers in its ears and >>>>>> is singing "La La La" rather loudly :) >>>>> >>>>> What about Google Code? Projects there can have multiple owners and >>>>> they support hg, have a tracker, and a wiki. >>>>> >>>>> >>>>>> >>>>>> Cheers, >>>>>> Nick. >>>>>> >>>>>> -- >>>>>> Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia >>>>>> >>>>> _______________________________________________ >>>>> Speed mailing list >>>>> Speed(a)python.org >>>>> http://mail.python.org/mailman/listinfo/speed >>>>> >>>> _______________________________________________ >>>> pypy-dev mailing list >>>> pypy-dev(a)python.org >>>> http://mail.python.org/mailman/listinfo/pypy-dev >>>> >>> >> _______________________________________________ >> Speed mailing list >> Speed(a)python.org >> http://mail.python.org/mailman/listinfo/speed >> > > > > -- > -------------------------------------------------- > Tennessee Leeuwenburg > http://myownhat.blogspot.com/ > "Don't believe everything you think"

2 1

Re: [Speed] [pypy-dev] Moving the project forward
by Tennessee Leeuwenburg 08 Sep '11

08 Sep '11

I would be more than happy to be progressing the cpython benchmarking automation, but I got no reply to my email summarising what needed to be done based on the comments from the list... I'm worried that anything I do will have to be undone if I don't follow the right path in terms of setting up buildbots, using Chef, etc etc. Thanks, -T On Fri, Sep 9, 2011 at 5:16 AM, Miquel Torres <tobami(a)googlemail.com> wrote: > Hi Jesse, > > Sorry, I spent last week polishing up some things to be able to > release Codespeed 0.9.1 and deploy that, because I had some Apache > configuration issues... > I'll have to try again this weekend. > > Cheers, > Miquel > > > 2011/9/8 Jesse Noller <jnoller(a)gmail.com>: >> PING: Did we make progress? >> >> On Thu, Sep 1, 2011 at 2:44 PM, Miquel Torres <tobami(a)googlemail.com> wrote: >>> You can also do that in Github, which I prefer. >>> >>> However, since CPython and PyPy use mercurial, the general preference >>> for Bitbucket is understandable. >>> >>> >>> 2011/9/1 Brett Cannon <brett(a)python.org>: >>>> On Thu, Sep 1, 2011 at 01:10, Nick Coghlan <ncoghlan(a)gmail.com> wrote: >>>>> On Thu, Sep 1, 2011 at 5:50 PM, Antonio Cuni <anto.cuni(a)gmail.com> wrote: >>>>>> On 31/08/11 22:11, Brett Cannon wrote: >>>>>>> >>>>>>> The PyPy folk could answer this as they have their repo on bitbucket >>>>>>> already. Else I guess we can just create a standalone account that >>>>>>> represents the official speed.python.org account. >>>>>> >>>>>> for pypy we do exactly that. There is a bitbucket user named "pypy" whose >>>>>> credentials are shared among all the core devs. >>>>> >>>>> The security auditing part of my brain has its fingers in its ears and >>>>> is singing "La La La" rather loudly :) >>>> >>>> What about Google Code? Projects there can have multiple owners and >>>> they support hg, have a tracker, and a wiki. >>>> >>>> >>>>> >>>>> Cheers, >>>>> Nick. >>>>> >>>>> -- >>>>> Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia >>>>> >>>> _______________________________________________ >>>> Speed mailing list >>>> Speed(a)python.org >>>> http://mail.python.org/mailman/listinfo/speed >>>> >>> _______________________________________________ >>> pypy-dev mailing list >>> pypy-dev(a)python.org >>> http://mail.python.org/mailman/listinfo/pypy-dev >>> >> > _______________________________________________ > Speed mailing list > Speed(a)python.org > http://mail.python.org/mailman/listinfo/speed > -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

1 0

Re: [Speed] [pypy-dev] Moving the project forward
by Miquel Torres 08 Sep '11

08 Sep '11

Hi Jesse, Sorry, I spent last week polishing up some things to be able to release Codespeed 0.9.1 and deploy that, because I had some Apache configuration issues... I'll have to try again this weekend. Cheers, Miquel 2011/9/8 Jesse Noller <jnoller(a)gmail.com>: > PING: Did we make progress? > > On Thu, Sep 1, 2011 at 2:44 PM, Miquel Torres <tobami(a)googlemail.com> wrote: >> You can also do that in Github, which I prefer. >> >> However, since CPython and PyPy use mercurial, the general preference >> for Bitbucket is understandable. >> >> >> 2011/9/1 Brett Cannon <brett(a)python.org>: >>> On Thu, Sep 1, 2011 at 01:10, Nick Coghlan <ncoghlan(a)gmail.com> wrote: >>>> On Thu, Sep 1, 2011 at 5:50 PM, Antonio Cuni <anto.cuni(a)gmail.com> wrote: >>>>> On 31/08/11 22:11, Brett Cannon wrote: >>>>>> >>>>>> The PyPy folk could answer this as they have their repo on bitbucket >>>>>> already. Else I guess we can just create a standalone account that >>>>>> represents the official speed.python.org account. >>>>> >>>>> for pypy we do exactly that. There is a bitbucket user named "pypy" whose >>>>> credentials are shared among all the core devs. >>>> >>>> The security auditing part of my brain has its fingers in its ears and >>>> is singing "La La La" rather loudly :) >>> >>> What about Google Code? Projects there can have multiple owners and >>> they support hg, have a tracker, and a wiki. >>> >>> >>>> >>>> Cheers, >>>> Nick. >>>> >>>> -- >>>> Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia >>>> >>> _______________________________________________ >>> Speed mailing list >>> Speed(a)python.org >>> http://mail.python.org/mailman/listinfo/speed >>> >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev(a)python.org >> http://mail.python.org/mailman/listinfo/pypy-dev >> >

1 0

Re: [Speed] [pypy-dev] Moving the project forward
by Jesse Noller 08 Sep '11

08 Sep '11

PING: Did we make progress? On Thu, Sep 1, 2011 at 2:44 PM, Miquel Torres <tobami(a)googlemail.com> wrote: > You can also do that in Github, which I prefer. > > However, since CPython and PyPy use mercurial, the general preference > for Bitbucket is understandable. > > > 2011/9/1 Brett Cannon <brett(a)python.org>: >> On Thu, Sep 1, 2011 at 01:10, Nick Coghlan <ncoghlan(a)gmail.com> wrote: >>> On Thu, Sep 1, 2011 at 5:50 PM, Antonio Cuni <anto.cuni(a)gmail.com> wrote: >>>> On 31/08/11 22:11, Brett Cannon wrote: >>>>> >>>>> The PyPy folk could answer this as they have their repo on bitbucket >>>>> already. Else I guess we can just create a standalone account that >>>>> represents the official speed.python.org account. >>>> >>>> for pypy we do exactly that. There is a bitbucket user named "pypy" whose >>>> credentials are shared among all the core devs. >>> >>> The security auditing part of my brain has its fingers in its ears and >>> is singing "La La La" rather loudly :) >> >> What about Google Code? Projects there can have multiple owners and >> they support hg, have a tracker, and a wiki. >> >> >>> >>> Cheers, >>> Nick. >>> >>> -- >>> Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia >>> >> _______________________________________________ >> Speed mailing list >> Speed(a)python.org >> http://mail.python.org/mailman/listinfo/speed >> > _______________________________________________ > pypy-dev mailing list > pypy-dev(a)python.org > http://mail.python.org/mailman/listinfo/pypy-dev >

1 0

Re: [Speed] Moving the project forward
by Stefan Behnel 02 Sep '11

02 Sep '11

Miquel Torres, 01.09.2011 20:44: > You can also do that in Github, which I prefer. > > However, since CPython and PyPy use mercurial, the general preference > for Bitbucket is understandable. Note that github is perfectly usable with hg, using hg-git. Stefan

1 0

Re: [Speed] [pypy-dev] Moving the project forward
by Miquel Torres 01 Sep '11

01 Sep '11

You can also do that in Github, which I prefer. However, since CPython and PyPy use mercurial, the general preference for Bitbucket is understandable. 2011/9/1 Brett Cannon <brett(a)python.org>: > On Thu, Sep 1, 2011 at 01:10, Nick Coghlan <ncoghlan(a)gmail.com> wrote: >> On Thu, Sep 1, 2011 at 5:50 PM, Antonio Cuni <anto.cuni(a)gmail.com> wrote: >>> On 31/08/11 22:11, Brett Cannon wrote: >>>> >>>> The PyPy folk could answer this as they have their repo on bitbucket >>>> already. Else I guess we can just create a standalone account that >>>> represents the official speed.python.org account. >>> >>> for pypy we do exactly that. There is a bitbucket user named "pypy" whose >>> credentials are shared among all the core devs. >> >> The security auditing part of my brain has its fingers in its ears and >> is singing "La La La" rather loudly :) > > What about Google Code? Projects there can have multiple owners and > they support hg, have a tracker, and a wiki. > > >> >> Cheers, >> Nick. >> >> -- >> Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia >> > _______________________________________________ > Speed mailing list > Speed(a)python.org > http://mail.python.org/mailman/listinfo/speed >

1 0

Re: [Speed] [pypy-dev] Moving the project forward
by Brett Cannon 01 Sep '11

01 Sep '11

On Thu, Sep 1, 2011 at 01:10, Nick Coghlan <ncoghlan(a)gmail.com> wrote: > On Thu, Sep 1, 2011 at 5:50 PM, Antonio Cuni <anto.cuni(a)gmail.com> wrote: >> On 31/08/11 22:11, Brett Cannon wrote: >>> >>> The PyPy folk could answer this as they have their repo on bitbucket >>> already. Else I guess we can just create a standalone account that >>> represents the official speed.python.org account. >> >> for pypy we do exactly that. There is a bitbucket user named "pypy" whose >> credentials are shared among all the core devs. > > The security auditing part of my brain has its fingers in its ears and > is singing "La La La" rather loudly :) What about Google Code? Projects there can have multiple owners and they support hg, have a tracker, and a wiki. > > Cheers, > Nick. > > -- > Nick Coghlan | ncoghlan(a)gmail.com | Brisbane, Australia >

1 0

Re: [Speed] [pypy-dev] Moving the project forward
by Antonio Cuni 01 Sep '11

01 Sep '11

On 31/08/11 22:11, Brett Cannon wrote: > The PyPy folk could answer this as they have their repo on bitbucket > already. Else I guess we can just create a standalone account that > represents the official speed.python.org account. for pypy we do exactly that. There is a bitbucket user named "pypy" whose credentials are shared among all the core devs. ciao, Anto

2 1

Re: [Speed] [pypy-dev] Co-ordinating benchmarking
by Tennessee Leeuwenburg 01 Sep '11

01 Sep '11

Okay, so all of a sudden there seem to be a *lot* of people looking at this. This was a long thread quickly, and I only just got up to speed with it. There are a lot of new names, and I don't know what existing skills, interest and territories exist. Apologies for any faux pas. I would like to organise the list of tasks a bit more clearly if that is okay. I may be less familiar with the parts of this process than others, so I just want to get it down clearly. What I've done: -- Clone the Python repo into /home/speedracer/cpython/cpython (updated to 2.7) -- Installed os packages to support a reasonable build. -- Built and installed python2.7 into /home/speedraver/cpython/27_bin Presumably, people would like to be monitoring both PyPy and Cpython as they progress over time. This means some kind of auto runner which updates from the repo, re-runs the timings, and submits them to codespeed. I am unclear on whether there is a "clear winner" for this piece of technology. List of Required Infrastructure on speed.python.org: -- A home for the cpython repo (check) -- An installed and running codespeed server (pending) -- A buildbot / automation for keeping up to date with the PyPy and cpython repos (???) -- Automation to execute a process which must (???) 1) Run the benchmarks 2) Submit results to codespeed I would suggest that the codespeed server be installed into speedracer's home directory. This must all be installable and configurable from Chef (which appears to me like Fabric, i.e. an automation tool for deployment and management of systems). This is not yet accomplished. We also clearly need some kind of wiki documentation on what is going on, so that contributors (or just newbies like me) can figure out where things are at and what is going on. The bitbucket project is great, but the task titles are currently rather brief if someone isn't already totally up-to-speed on what is going on. There appear to me to be two unresolved questions: 1) What piece of technology should we use for a buildbot / build automation 2) What piece of technology should we use for the benchmark runner? I have no suggestions on (1), it's not a strong point for me. As regards (2), I am ignorant to what others might already be using, except to say the landscape seems unclear and fractured to me. My work, benchmarker.py, is likely to be adaptable to our needs and I am more than happy to support the package so it can be applied here. As I understand it, the GSOC project was about the actual benchmarking functions, not so much about automation and support for managing the results of benchmarking. If an already-working alternative to benchmarker.py exists and makes more sense to use, then that is fine by me. I would still like to help out to learn more about benchmarking. The main issue with me as a contributor will be time. I have a full plate as a result of Real Life, so I will sometimes go dark for a week or so. However, I'm motivated and interested, and can put in a few hours a week most weeks. Do I have this right? Is that a reasonable description of the work breakdown? Do we have clear names against tasks so that co-ordination can be done through those people (rather than via the whole list)? Regards, -Tennessee On Thu, Sep 1, 2011 at 6:28 AM, Noah Kantrowitz <noah(a)coderanger.net> wrote: > Its all branches all the way down, so we can start work anywhere and push it to an "official" PSF bin later I think. I'm sure we will want to host a mirror of it on the python.org hg server too, just for discoverability. > > --Noah > > On Aug 31, 2011, at 1:12 PM, Miquel Torres wrote: > >> Oh, cool, so there will be an Opscode hosted account for the PSF, >> right? Then the Chef repo should be for the PSF. Maybe in a current >> account somewhere? What do you propose? >> >> Miquel >> >> >> 2011/8/31 Noah Kantrowitz <noah(a)coderanger.net>: >>> Opscode has already agreed to donate a Hosted account as long we keep it under ~20 clients :-) I can hand out the info for it to anyone that wants. As for setting up the Chef repo, just remember we are trying to not manage this system in isolation and that it will be part of a bigger PSF infrastructure management effort. >>> >>> --Noah >>> >>> On Aug 31, 2011, at 11:34 AM, Miquel Torres wrote: >>> >>>> Hi all, >>>> >>>> though I took up on the task of installing a Codespeed instance >>>> myself, I didn't have time until now. This weekend I will definitely >>>> have a *lot* of time to work on this, so count on that task being >>>> done by then. >>>> >>>> The bitbucket issue tracker is a start (though a organization account >>>> would be better) and the splash page is great. So let's get started >>>> organizing things. >>>> >>>> Regarding the deployment strategy, it turns out I use Chef at work, so >>>> I am in full agreement with Noah here (yey!). Actually, I am the >>>> author of LittleChef (which we can use as a tool to execute Chef on >>>> the node). >>>> >>>> So, Configuration Management. I would propose that Noah starts the >>>> repo with the Chef cookbooks (preferably a complete LittleChef >>>> kitchen, but that is not a must :), and gets the main recipes (apache, >>>> django) going, while I create a cookbook for Codespeed. What do you >>>> think? >>>> >>>> The benchmark runner question is still open. We need to clarify that. >>>> Use the pypy runner? Tennessee's work? >>>> >>>> Regarding repositories and issues, we could maybe have a "speed" >>>> organization account (not sure on Bitbucket, you can do that in >>>> Github), where we have a wiki, issues, and runner + config management >>>> repo + other stuff. >>>> >>>> Cheers, >>>> Miquel >>>> >>>> 2011/8/31 Jesse Noller <jnoller(a)gmail.com>: >>>>> I've put up a splash page for the project this AM: >>>>> >>>>> http://speed.python.org/ >>>>> >>>>> jesse >>>>> _______________________________________________ >>>>> pypy-dev mailing list >>>>> pypy-dev(a)python.org >>>>> http://mail.python.org/mailman/listinfo/pypy-dev >>>>> >>>> _______________________________________________ >>>> Speed mailing list >>>> Speed(a)python.org >>>> http://mail.python.org/mailman/listinfo/speed >>> >>> > > > _______________________________________________ > Speed mailing list > Speed(a)python.org > http://mail.python.org/mailman/listinfo/speed > > -- -------------------------------------------------- Tennessee Leeuwenburg http://myownhat.blogspot.com/ "Don't believe everything you think"

1 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

Speed September 2011