Mailman 3 August 2011 - Speed

Re: [Speed] [pypy-dev] Moving the project forward
by Noah Kantrowitz Aug. 31, 2011

Aug. 31, 2011

Its all branches all the way down, so we can start work anywhere and push it to an "official" PSF bin later I think. I'm sure we will want to host a mirror of it on the python.org hg server too, just for discoverability. --Noah On Aug 31, 2011, at 1:12 PM, Miquel Torres wrote: > Oh, cool, so there will be an Opscode hosted account for the PSF, > right? Then the Chef repo should be for the PSF. Maybe in a current > account somewhere? What do you propose? > > Miquel > > > 2011/8/31 Noah Kantrowitz <noah(a)coderanger.net>: >> Opscode has already agreed to donate a Hosted account as long we keep it under ~20 clients :-) I can hand out the info for it to anyone that wants. As for setting up the Chef repo, just remember we are trying to not manage this system in isolation and that it will be part of a bigger PSF infrastructure management effort. >> >> --Noah >> >> On Aug 31, 2011, at 11:34 AM, Miquel Torres wrote: >> >>> Hi all, >>> >>> though I took up on the task of installing a Codespeed instance >>> myself, I didn't have time until now. This weekend I will definitely >>> have a *lot* of time to work on this, so count on that task being >>> done by then. >>> >>> The bitbucket issue tracker is a start (though a organization account >>> would be better) and the splash page is great. So let's get started >>> organizing things. >>> >>> Regarding the deployment strategy, it turns out I use Chef at work, so >>> I am in full agreement with Noah here (yey!). Actually, I am the >>> author of LittleChef (which we can use as a tool to execute Chef on >>> the node). >>> >>> So, Configuration Management. I would propose that Noah starts the >>> repo with the Chef cookbooks (preferably a complete LittleChef >>> kitchen, but that is not a must :), and gets the main recipes (apache, >>> django) going, while I create a cookbook for Codespeed. What do you >>> think? >>> >>> The benchmark runner question is still open. We need to clarify that. >>> Use the pypy runner? Tennessee's work? >>> >>> Regarding repositories and issues, we could maybe have a "speed" >>> organization account (not sure on Bitbucket, you can do that in >>> Github), where we have a wiki, issues, and runner + config management >>> repo + other stuff. >>> >>> Cheers, >>> Miquel >>> >>> 2011/8/31 Jesse Noller <jnoller(a)gmail.com>: >>>> I've put up a splash page for the project this AM: >>>> >>>> http://speed.python.org/ >>>> >>>> jesse >>>> _______________________________________________ >>>> pypy-dev mailing list >>>> pypy-dev(a)python.org >>>> http://mail.python.org/mailman/listinfo/pypy-dev >>>> >>> _______________________________________________ >>> Speed mailing list >>> Speed(a)python.org >>> http://mail.python.org/mailman/listinfo/speed >> >>

1 0

Re: [Speed] [pypy-dev] Moving the project forward
by Noah Kantrowitz Aug. 31, 2011

Aug. 31, 2011

Opscode has already agreed to donate a Hosted account as long we keep it under ~20 clients :-) I can hand out the info for it to anyone that wants. As for setting up the Chef repo, just remember we are trying to not manage this system in isolation and that it will be part of a bigger PSF infrastructure management effort. --Noah On Aug 31, 2011, at 11:34 AM, Miquel Torres wrote: > Hi all, > > though I took up on the task of installing a Codespeed instance > myself, I didn't have time until now. This weekend I will definitely > have a *lot* of time to work on this, so count on that task being > done by then. > > The bitbucket issue tracker is a start (though a organization account > would be better) and the splash page is great. So let's get started > organizing things. > > Regarding the deployment strategy, it turns out I use Chef at work, so > I am in full agreement with Noah here (yey!). Actually, I am the > author of LittleChef (which we can use as a tool to execute Chef on > the node). > > So, Configuration Management. I would propose that Noah starts the > repo with the Chef cookbooks (preferably a complete LittleChef > kitchen, but that is not a must :), and gets the main recipes (apache, > django) going, while I create a cookbook for Codespeed. What do you > think? > > The benchmark runner question is still open. We need to clarify that. > Use the pypy runner? Tennessee's work? > > Regarding repositories and issues, we could maybe have a "speed" > organization account (not sure on Bitbucket, you can do that in > Github), where we have a wiki, issues, and runner + config management > repo + other stuff. > > Cheers, > Miquel > > 2011/8/31 Jesse Noller <jnoller(a)gmail.com>: >> I've put up a splash page for the project this AM: >> >> http://speed.python.org/ >> >> jesse >> _______________________________________________ >> pypy-dev mailing list >> pypy-dev(a)python.org >> http://mail.python.org/mailman/listinfo/pypy-dev >> > _______________________________________________ > Speed mailing list > Speed(a)python.org > http://mail.python.org/mailman/listinfo/speed

2 1

Re: [Speed] [pypy-dev] Moving the project forward
by Brett Cannon Aug. 31, 2011

Aug. 31, 2011

On Wed, Aug 31, 2011 at 11:34, Miquel Torres <tobami(a)googlemail.com> wrote: > Hi all, > > though I took up on the task of installing a Codespeed instance > myself, I didn't have time until now. This weekend I will definitely > have a *lot* of time to work on this, so count on that task being > done by then. > > The bitbucket issue tracker is a start (though a organization account > would be better) and the splash page is great. So let's get started > organizing things. [SNIP] > Regarding repositories and issues, we could maybe have a "speed" > organization account (not sure on Bitbucket, you can do that in > Github), where we have a wiki, issues, and runner + config management > repo + other stuff. The PyPy folk could answer this as they have their repo on bitbucket already. Else I guess we can just create a standalone account that represents the official speed.python.org account.

1 0

Re: [Speed] [pypy-dev] Moving the project forward
by Miquel Torres Aug. 31, 2011

Aug. 31, 2011

Hi all, though I took up on the task of installing a Codespeed instance myself, I didn't have time until now. This weekend I will definitely have a *lot* of time to work on this, so count on that task being done by then. The bitbucket issue tracker is a start (though a organization account would be better) and the splash page is great. So let's get started organizing things. Regarding the deployment strategy, it turns out I use Chef at work, so I am in full agreement with Noah here (yey!). Actually, I am the author of LittleChef (which we can use as a tool to execute Chef on the node). So, Configuration Management. I would propose that Noah starts the repo with the Chef cookbooks (preferably a complete LittleChef kitchen, but that is not a must :), and gets the main recipes (apache, django) going, while I create a cookbook for Codespeed. What do you think? The benchmark runner question is still open. We need to clarify that. Use the pypy runner? Tennessee's work? Regarding repositories and issues, we could maybe have a "speed" organization account (not sure on Bitbucket, you can do that in Github), where we have a wiki, issues, and runner + config management repo + other stuff. Cheers, Miquel 2011/8/31 Jesse Noller <jnoller(a)gmail.com>: > I've put up a splash page for the project this AM: > > http://speed.python.org/ > > jesse > _______________________________________________ > pypy-dev mailing list > pypy-dev(a)python.org > http://mail.python.org/mailman/listinfo/pypy-dev >

1 0

Re: [Speed] Moving the project forward
by Jesse Noller Aug. 31, 2011

Aug. 31, 2011

I've put up a splash page for the project this AM: http://speed.python.org/ jesse

1 0

Re: [Speed] [pypy-dev] Moving the project forward
by Laura Creighton Aug. 31, 2011

Aug. 31, 2011

I want to know the status of DasIch's GSoC project to work on the coderunner. i.e. GSoC is over now, so we can start hacking on whereever he left off, correct? Laura

2 1

Re: [Speed] [pypy-dev] Moving the project forward
by Jeroen Ruigrok van der Werven Aug. 31, 2011

Aug. 31, 2011

-On [20110831 12:22], Nick Coghlan (ncoghlan(a)gmail.com) wrote: >So I'll ask a (deceptively) simple question: who's going to set up a >BitBucket project to use as the issue tracker for speed.python.org (at >least initially), populate it with information about the current plans >for the speed.python.org site and then announce that to this list? Something like: https://bitbucket.org/asmodai/python-speed/issues?status=new&status=open ? -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai イェルーンラウフロックヴァンデルウェルヴェン http://www.in-nomine.org/ | GPG: 2EAC625B Take thy beak from out my heart and take thy form from off my door!

2 1

Re: [Speed] [pypy-dev] Moving the project forward
by Noah Kantrowitz Aug. 31, 2011

Aug. 31, 2011

Yahr, I be here. I would really like to see this done under a config management system (I prefer Chef and thats been the plan so far unless there are heavy objections). In general no one should ever be changing things on any PSF server by hand if at all possible in the interests of disaster recovery, reproducibility, and some modicum of enforced documentation (even if that doc is just a Chef recipe). --Noah On Aug 30, 2011, at 5:35 PM, Alex Gaynor wrote: > I've gotten Noah Kantrowitz on here, he's doing some other infrastructure stuff for the PSF, and would like to include speed.python.org in the proper organization of the machines, rather than ad-hoc "everyone installs what they think it needs" :) > > Alex > > On Tue, Aug 30, 2011 at 8:28 PM, Jesse Noller <jnoller(a)gmail.com> wrote: > (Re-sending intentionally - I wasn't subbed to pypy-dev and got rejected) > > On Tue, Aug 30, 2011 at 8:22 PM, Jesse Noller <jnoller(a)gmail.com> wrote: > > Here's a summary (nicely put together by Nick Coghlan): > > > > - OSU/OSL have set up the machine itself (details in the July list archives) > > - I am the machine admin (I have root) > > - Tennessee and Miquel are working on getting codespeed up and running > > > > - I and Maciej are already too busy to handle the coordination/admin > > side of things > > > > I just added the "speedracer" account - a common account to setup > > things as needed. Currently, I don't see activity from Miquel on > > setting up the codespeed instance. I know Tennessee has been looking > > at setting up the runner. > > > > What we need is this, very simply: > > > > 1 - A working, running codespeed instance installed > > 2 - Benchmark runner(s) setup and going - for PyPy and CPython 2.x > > since the benchmarks don't run on Python 3 yet. > > > > > > Other needs/notes: > > - a project issue tracker doesn't appear to exist yet (needs to be done) > > - a better 'placeholder/about' page would be useful that offered links to: > > - the speed(a)python.org mailing list signup page > > - the issue tracker (once it exists) > > - the codespeed repo > > > > Ideally, the front landing page would end up looking like: > > http://speed.pypy.org/ > > > > - the aim is to eventually have the web front end running on different > > (but still OSU/OSL hosted) hardware, although that likely isn't > > critical right now > > > > - the current 5 GB /tmp size may be a problem for PyPy > > > > Where do we go from here - I can not be the one to keep pestering the > > group to make forward progress. On behalf of the PSF, and the CPython > > and PyPy teams, I got us a donated platform for running these shared, > > common benchmarks. Right now we have a 10,000$+ paper weight taking > > rack space in OSU/OSL. > > > > I need someone to take lead on this and get the project moving forward > > - as a reminder, here was the original proposal I made to the board, > > after PyCon 2011: > > > > """ > > Coming out of the PyCon VM and language summits, it was commonly > > agreed that PyPy, CPython, IronPython and Jython should strive to move > > to a common set of benchmarks and a single performance-oriented site. > > This was agreed upon by the maintainers present at the conference, and > > included leads from all of the major implementations. This project > > would be lead by the various VM development teams, and based around > > the Unladen Swallow/PyPy benchmark suite. The site would be derived > > from: > > > > http://speed.pypy.org/ > > > > There are already GSOC students potentially lined up to work on > > porting the test suite to Python 3! > > > > There, of course, is the requirement that we have: > > > > 1> A machine > > 2> Hosting for that machine > > > > And I took the job of finding both. As the OSU/OSL [1] has come up > > several times in discussions about free, monitored hosting, I felt > > that now would be the time to float the trial balloon and start a > > continuing relationship with the OSU/OSL team. I got to speak with > > some of the team at PyCon, and I was quite impressed with their > > enthusiasm and willingness to help out. > > """ > > > > I need help from everyone to get this up and running: the PyPy team is > > most familiar with the speed.pypy.org/codespeed system. If we could > > get a dump of what needs to be done to get things setup, that may help > > get more hands involved. > > > > I am feeling personally responsible that this has not been moving > > forward - I need your help to get this up and running for the > > betterment of CPython and PyPy. This project benefits both projects, > > and Python in general immensely. > > > > Once up and running, this system will be prominently placed on the > > Python.org home page, and be used by potential users to help select > > and evaluate their Python runtime. > > > > Jesse > > > _______________________________________________ > pypy-dev mailing list > pypy-dev(a)python.org > http://mail.python.org/mailman/listinfo/pypy-dev > > > > -- > "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) > "The people's good is the highest law." -- Cicero > > _______________________________________________ > Speed mailing list > Speed(a)python.org > http://mail.python.org/mailman/listinfo/speed

4 3

Re: [Speed] [pypy-dev] Moving the project forward
by Antonio Cuni Aug. 31, 2011

Aug. 31, 2011

Hi Jesse, hi all, On 31/08/11 02:37, Jesse Noller wrote: > Yes, I should have looped Noah in sooner. > > I have all the keys / passwords. Right now we need: > > 1. codespeed > 2. benchmark runners The second item is divided into two sub-items: the runner itself, and something that triggers a run nightly/on request. In PyPy, we currently use a modified version of the Unladen Swallow benchmark runner (look at runner.py): https://bitbucket.org/pypy/benchmarks/src in its current state, it is sub-optimal, because it was designed to compare two pythons side-by-side, and not to save the results and then upload them to a website. As a consequence, it always runs the benchmarks twice. With PyPy, it's lesser of a problem because for each binary we always run the benchmarks with the default options and with "--jit off" anyway, but for CPython running them twice would result in a waste of time. I suppose that the current runner is good enough for now and to get started, however at some point it will need some care. Then, we need something which triggers the actual benchmark run. In PyPy, we use buildbot, and each benchmark run first do a full translation, then executes the benchmarks. I can setup a buildbot instance on speed.pypy.org, if you give me access to the machine. I propose that as a very first step, we just make speed.pypy.org a buildslave which depends on pypy's own buildmaster. This makes it very easy and fast to setup it, so we can have something running soon. ciao, Anto

1 1

Re: [Speed] [pypy-dev] Moving the project forward
by Noah Kantrowitz Aug. 31, 2011

Aug. 31, 2011

Is there an a readme for installing those somewhere? If someone can show me the commands a human would run I can handle scripting them :-) --Noah On Aug 30, 2011, at 5:37 PM, Jesse Noller wrote: > Yes, I should have looped Noah in sooner. > > I have all the keys / passwords. Right now we need: > > 1. codespeed > 2. benchmark runners > > I've installed a very basic system with Django, apache, mod_wsgi > > On Tue, Aug 30, 2011 at 8:35 PM, Alex Gaynor <alex.gaynor(a)gmail.com> wrote: >> I've gotten Noah Kantrowitz on here, he's doing some other infrastructure >> stuff for the PSF, and would like to include speed.python.org in the proper >> organization of the machines, rather than ad-hoc "everyone installs what >> they think it needs" :) >> Alex >> >> On Tue, Aug 30, 2011 at 8:28 PM, Jesse Noller <jnoller(a)gmail.com> wrote: >>> >>> (Re-sending intentionally - I wasn't subbed to pypy-dev and got rejected) >>> >>> On Tue, Aug 30, 2011 at 8:22 PM, Jesse Noller <jnoller(a)gmail.com> wrote: >>>> Here's a summary (nicely put together by Nick Coghlan): >>>> >>>> - OSU/OSL have set up the machine itself (details in the July list >>>> archives) >>>> - I am the machine admin (I have root) >>>> - Tennessee and Miquel are working on getting codespeed up and running >>>> >>>> - I and Maciej are already too busy to handle the coordination/admin >>>> side of things >>>> >>>> I just added the "speedracer" account - a common account to setup >>>> things as needed. Currently, I don't see activity from Miquel on >>>> setting up the codespeed instance. I know Tennessee has been looking >>>> at setting up the runner. >>>> >>>> What we need is this, very simply: >>>> >>>> 1 - A working, running codespeed instance installed >>>> 2 - Benchmark runner(s) setup and going - for PyPy and CPython 2.x >>>> since the benchmarks don't run on Python 3 yet. >>>> >>>> >>>> Other needs/notes: >>>> - a project issue tracker doesn't appear to exist yet (needs to be done) >>>> - a better 'placeholder/about' page would be useful that offered links >>>> to: >>>> - the speed(a)python.org mailing list signup page >>>> - the issue tracker (once it exists) >>>> - the codespeed repo >>>> >>>> Ideally, the front landing page would end up looking like: >>>> http://speed.pypy.org/ >>>> >>>> - the aim is to eventually have the web front end running on different >>>> (but still OSU/OSL hosted) hardware, although that likely isn't >>>> critical right now >>>> >>>> - the current 5 GB /tmp size may be a problem for PyPy >>>> >>>> Where do we go from here - I can not be the one to keep pestering the >>>> group to make forward progress. On behalf of the PSF, and the CPython >>>> and PyPy teams, I got us a donated platform for running these shared, >>>> common benchmarks. Right now we have a 10,000$+ paper weight taking >>>> rack space in OSU/OSL. >>>> >>>> I need someone to take lead on this and get the project moving forward >>>> - as a reminder, here was the original proposal I made to the board, >>>> after PyCon 2011: >>>> >>>> """ >>>> Coming out of the PyCon VM and language summits, it was commonly >>>> agreed that PyPy, CPython, IronPython and Jython should strive to move >>>> to a common set of benchmarks and a single performance-oriented site. >>>> This was agreed upon by the maintainers present at the conference, and >>>> included leads from all of the major implementations. This project >>>> would be lead by the various VM development teams, and based around >>>> the Unladen Swallow/PyPy benchmark suite. The site would be derived >>>> from: >>>> >>>> http://speed.pypy.org/ >>>> >>>> There are already GSOC students potentially lined up to work on >>>> porting the test suite to Python 3! >>>> >>>> There, of course, is the requirement that we have: >>>> >>>> 1> A machine >>>> 2> Hosting for that machine >>>> >>>> And I took the job of finding both. As the OSU/OSL [1] has come up >>>> several times in discussions about free, monitored hosting, I felt >>>> that now would be the time to float the trial balloon and start a >>>> continuing relationship with the OSU/OSL team. I got to speak with >>>> some of the team at PyCon, and I was quite impressed with their >>>> enthusiasm and willingness to help out. >>>> """ >>>> >>>> I need help from everyone to get this up and running: the PyPy team is >>>> most familiar with the speed.pypy.org/codespeed system. If we could >>>> get a dump of what needs to be done to get things setup, that may help >>>> get more hands involved. >>>> >>>> I am feeling personally responsible that this has not been moving >>>> forward - I need your help to get this up and running for the >>>> betterment of CPython and PyPy. This project benefits both projects, >>>> and Python in general immensely. >>>> >>>> Once up and running, this system will be prominently placed on the >>>> Python.org home page, and be used by potential users to help select >>>> and evaluate their Python runtime. >>>> >>>> Jesse >>>> >>> _______________________________________________ >>> pypy-dev mailing list >>> pypy-dev(a)python.org >>> http://mail.python.org/mailman/listinfo/pypy-dev >> >> >> >> -- >> "I disapprove of what you say, but I will defend to the death your right to >> say it." -- Evelyn Beatrice Hall (summarizing Voltaire) >> "The people's good is the highest law." -- Cicero >> >> > _______________________________________________ > Speed mailing list > Speed(a)python.org > http://mail.python.org/mailman/listinfo/speed

1 0