Catchy subject eh? Unless, of course, you've seen Forgetting Sarah
Marshall, in which case, it's probably a tad disturbing.
Nevertheless, I do have a surprise for everyone. I spent a lot of
time early last year trying to keep the buildbots green, especially
the x64 Windows ones. This was quite enjoyable, at least initially.
By the time I got back to London after PyCon last year, the buildbot
honeymoon was wearing off. As there's usually only one person that
has access to a given buildbot, and that person is rarely you, it
can be a right pain in the ass trying to debug problems you can't
reproduce on platforms you don't have access to (especially if all
you've got is an error message that doesn't make sense).
I was convinced there must be a better way.
Around mid-April, the buildbot on my FreeBSD 6.x box I had recently
set up kept failing on on setitimer tests. I received an email from
a chap named Guilherme Polo who had seen the buildbot test failures
and wanted to assist. I was swamped with client work at the time; I
wasn't able to run any of the test scripts he sent me, but, hey, he
seemed like a nice chap, so I thought what the hell, and just gave
him an account on the box and checked out a copy of trunk in his
home directory.
Now, mind you, this was before Guilherme was a committer; I didn't
know the guy from a bar of soap. For all I knew, he could have been
using the shell account to launch a massive DDoS or phishing scam.
However, no more than five minutes after I created his account, he'd
diagnosed the problem, replicated the behaviour in a C program, and,
with a bit of googling, figured out what was wrong and proposed a
fix.
And that's when it hit me. Buildbots are fine when everything is
running smoothly, but nothing compares to actually having access to
a system when you're trying to debug something.
So, I thought to myself, why not buy a couple of clunky old boxes
off eBay and donate them to the PSF, such that all developers had
access to them? I dropped a note to Guido and Neal, they put me
in touch with Titus, who had just accepted a position at Michigan
State, and, well...
Ten months, seven trips to MSU, six blown fuses and about $60,000
later, I'm proud to introduce you all to Snakebite: The Open Network!
A network of around 37-ish servers of all different shapes and sizes,
spread over three sites, specifically geared towards the needs of
open source projects like Python.
Every CPython, Jython, IronPython and PyPy committer will have access
to every development server on the network. I've also extended the
offer to prominent Python projects like Django and Twisted.
Eventually, I'll invite other open source projects to participate
(Apache, Subversion, MySQL, Postgres, etc), but the network is my
gift to All Things Python, first and foremost, so Python projects
will always get preferential treatment.
Support for the initiative so far has been nothing short of sublime.
Microsoft jumped on board and provided unlimited MSDN licenses in
less time than it took me to write them an e-mail asking for stuff.
I sent HP an e-mail asking if they could spare a Tru64 license, and
maybe 2GB of RAM for an extremely crappy Itanium box I bought off
eBay. They saw my Tru64 license request and raised with media and
licenses to the latest version of HP-UX.
Unfortunately, it was too much trouble for them to try and source
2GB of RAM for the Itanium I bought. So, instead, they shipped two
massive quad Itanium 2 RX-5670s, chock full of 73GB 15k disks and
no less than 78GB of RAM between the two servers; 32GB in one and
46GB in the other. Well then.
(I'd hate to think what would have turned up had I asked them for
two quad Itanium monsters.)
Sun, Google and Canonical have also expressed a lot of interest in
the project -- I stopped asking for hardware a while back though as
we've literally run out of space to host it all.
The website is live, but the content is a bit sparse at the moment,
excluding the poorly worded front page and the reasonably accurate
network page:
http://www.snakebite.org
http://www.snakebite.org/network
It'll probably be a few weeks before you can start logging in and
doing stuff. The HPCC/CSE server room at MSU is about to have walls
knocked in and ramps built in order to accommodate a giant PDU that
has been sitting outside it for about six months; the Snakebite rack
is going to get shuffled around a bit so I figure there's not much
point going live before that's taken care of.
Other than that, I'm just happy to get this off my chest, ten months
is a freakin' long time to try and keep something like this a secret
;-)
Thanks to everyone who has contributed so far. Titus, Bill, Adam,
Kelly, George: Snakebite wouldn't be anything more than a twinkle
in my eye if it weren't for the support MSU has thrown behind the
project, thank you for all your efforts to date. Hank, Garrett and
Sam: having the support of Microsoft from very early on has been a
huge boost and the MSDN licenses have already been invaluable. Bob,
well, what can I say, there was a period there where every e-mail
thread between us seemed to result in something being shipped from
HP to MSU; thanks to you and HP's open source labs.
And last but not least, thanks to Guido and all the Python committers
for their tireless efforts to date. Although the sheer elegance of
the language is what initially attracted me to Python, it was the
developer community that made me want to stick around.
Snakebite is my gift to you!
Trent.
Trent,
Congrats on the current state of the project, you went quite a ways in a very short amount of time.
And I am glad we were able to assist you.
Thanks,
Hank Janssen Principal Group Program Manager/Director Microsoft Open Source Technology Center Desk: 425-706-2305 hjanssen@microsoft.com http://port25.technet.com
-----Original Message----- From: Trent Nelson [mailto:python-committers-list@trentnelson.com] Sent: Monday, January 26, 2009 3:33 PM To: python-committers@python.org Cc: bob.gobeille@hp.com; titus@idyll.org; glynn.foster@sun.com; jim.walker@sun.com; ted.leung@sun.com; Frank.Wierzbicki@Sun.COM; Hank Janssen; Garrett Serack; punch@cse.msu.edu; Sam Ramji; Chip Norkus; Dino Viehland; Jimmy Schementi; Dave Fugate; stockman@cse.msu.edu; pitcher2@cse.msu.edu; tnelson@onresolve.com; mark@canonical.com; exarkun@divmod.com; gylph@divmod.com; lhawthorn@google.com; doko@ubuntu.com; fuzzyman@voidspace.org.uk; oubiwann@divmod.com; jacob.kaplanmoss@gmail.com; psf@python.org; climber@cse.msu.edu Subject: I've got a surprise for you!
Catchy subject eh? Unless, of course, you've seen Forgetting Sarah
Marshall, in which case, it's probably a tad disturbing.
Nevertheless, I do have a surprise for everyone. I spent a lot of
time early last year trying to keep the buildbots green, especially
the x64 Windows ones. This was quite enjoyable, at least initially.
By the time I got back to London after PyCon last year, the buildbot
honeymoon was wearing off. As there's usually only one person that
has access to a given buildbot, and that person is rarely you, it
can be a right pain in the ass trying to debug problems you can't
reproduce on platforms you don't have access to (especially if all
you've got is an error message that doesn't make sense).
I was convinced there must be a better way.
Around mid-April, the buildbot on my FreeBSD 6.x box I had recently
set up kept failing on on setitimer tests. I received an email from
a chap named Guilherme Polo who had seen the buildbot test failures
and wanted to assist. I was swamped with client work at the time; I
wasn't able to run any of the test scripts he sent me, but, hey, he
seemed like a nice chap, so I thought what the hell, and just gave
him an account on the box and checked out a copy of trunk in his
home directory.
Now, mind you, this was before Guilherme was a committer; I didn't
know the guy from a bar of soap. For all I knew, he could have been
using the shell account to launch a massive DDoS or phishing scam.
However, no more than five minutes after I created his account, he'd
diagnosed the problem, replicated the behaviour in a C program, and,
with a bit of googling, figured out what was wrong and proposed a
fix.
And that's when it hit me. Buildbots are fine when everything is
running smoothly, but nothing compares to actually having access to
a system when you're trying to debug something.
So, I thought to myself, why not buy a couple of clunky old boxes
off eBay and donate them to the PSF, such that all developers had
access to them? I dropped a note to Guido and Neal, they put me
in touch with Titus, who had just accepted a position at Michigan
State, and, well...
Ten months, seven trips to MSU, six blown fuses and about $60,000
later, I'm proud to introduce you all to Snakebite: The Open Network!
A network of around 37-ish servers of all different shapes and sizes,
spread over three sites, specifically geared towards the needs of
open source projects like Python.
Every CPython, Jython, IronPython and PyPy committer will have access
to every development server on the network. I've also extended the
offer to prominent Python projects like Django and Twisted.
Eventually, I'll invite other open source projects to participate
(Apache, Subversion, MySQL, Postgres, etc), but the network is my
gift to All Things Python, first and foremost, so Python projects
will always get preferential treatment.
Support for the initiative so far has been nothing short of sublime.
Microsoft jumped on board and provided unlimited MSDN licenses in
less time than it took me to write them an e-mail asking for stuff.
I sent HP an e-mail asking if they could spare a Tru64 license, and
maybe 2GB of RAM for an extremely crappy Itanium box I bought off
eBay. They saw my Tru64 license request and raised with media and
licenses to the latest version of HP-UX.
Unfortunately, it was too much trouble for them to try and source
2GB of RAM for the Itanium I bought. So, instead, they shipped two
massive quad Itanium 2 RX-5670s, chock full of 73GB 15k disks and
no less than 78GB of RAM between the two servers; 32GB in one and
46GB in the other. Well then.
(I'd hate to think what would have turned up had I asked them for
two quad Itanium monsters.)
Sun, Google and Canonical have also expressed a lot of interest in
the project -- I stopped asking for hardware a while back though as
we've literally run out of space to host it all.
The website is live, but the content is a bit sparse at the moment,
excluding the poorly worded front page and the reasonably accurate
network page:
http://www.snakebite.org
http://www.snakebite.org/network
It'll probably be a few weeks before you can start logging in and
doing stuff. The HPCC/CSE server room at MSU is about to have walls
knocked in and ramps built in order to accommodate a giant PDU that
has been sitting outside it for about six months; the Snakebite rack
is going to get shuffled around a bit so I figure there's not much
point going live before that's taken care of.
Other than that, I'm just happy to get this off my chest, ten months
is a freakin' long time to try and keep something like this a secret
;-)
Thanks to everyone who has contributed so far. Titus, Bill, Adam,
Kelly, George: Snakebite wouldn't be anything more than a twinkle
in my eye if it weren't for the support MSU has thrown behind the
project, thank you for all your efforts to date. Hank, Garrett and
Sam: having the support of Microsoft from very early on has been a
huge boost and the MSDN licenses have already been invaluable. Bob,
well, what can I say, there was a period there where every e-mail
thread between us seemed to result in something being shipped from
HP to MSU; thanks to you and HP's open source labs.
And last but not least, thanks to Guido and all the Python committers
for their tireless efforts to date. Although the sheer elegance of
the language is what initially attracted me to Python, it was the
developer community that made me want to stick around.
Snakebite is my gift to you!
Trent.
[SNIP - Trent finally gets to announce Snakebite publicly]
Thanks for spear-heading this, Trent, and to everyone who helped along the way! This is an amazing project. It makes me actually want to try to fix a bug just to give it a go. This is definitely going to help keep Python ahead of the pack when it comes to cross-platform support and code quality.
I only have two questions at this point. One, is it cool to blog about this? And two, are there plans to have buildbots running on the machines at all so we know exactly which ones are triggering failures? Because it would be beyond cool to have a massive buildbot page where if I see something red on it I can just SSH directly into that machine and get the thing green.
-Brett
Trent Nelson wrote:
Thanks to everyone who has contributed so far. Titus, Bill, Adam, Kelly, George: Snakebite wouldn't be anything more than a twinkle in my eye if it weren't for the support MSU has thrown behind the project, thank you for all your efforts to date. Hank, Garrett and Sam: having the support of Microsoft from very early on has been a huge boost and the MSDN licenses have already been invaluable. Bob, well, what can I say, there was a period there where every e-mail thread between us seemed to result in something being shipped from HP to MSU; thanks to you and HP's open source labs. And last but not least, thanks to Guido and all the Python committers for their tireless efforts to date. Although the sheer elegance of the language is what initially attracted me to Python, it was the developer community that made me want to stick around. Snakebite is my gift to you!
Trent:
May I second all those thanks on behalf of the whole Python community. It's heart-warming to see support like this from so many directions.
A fantastic culmination to almost a year of hard work. Glad we don't have to keep it secret any more!
Thanks very much. I can't wait to start logging in ...
regards Steve
Steve Holden +1 571 484 6266 +1 800 494 3119 Chairman, Python Software Foundation www.python.org
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jan 26, 2009, at 6:32 PM, Trent Nelson wrote:
Ten months, seven trips to MSU, six blown fuses and about $60,000 later, I'm proud to introduce you all to Snakebite: The Open
Network! A network of around 37-ish servers of all different shapes and
sizes, spread over three sites, specifically geared towards the needs of open source projects like Python.
Congratulations Trent! I'm glad to see this finally come to fruition
and I can't wait to get a login :)
- -Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin)
iQCVAwUBSX5ny3EjvBPtnXfVAQIKfwP9HnrL9//KaW8BCr+L7zC5pDfteTnau8gt scGKT5hWtrFgPKVZqzXus7qNKUynt3Lc39GbTn/SN+YfQmgyWY2LN2T8Y3cE2SA/ 9YjHKSQRkbezeD34JU2Lnl0a9CT9bnmvftg0q7HEB1l9PJaa/b0taCL4nzeI6RUY Vv+THFZZMlg= =BYYI -----END PGP SIGNATURE-----
Trent Nelson schrieb:
Catchy subject eh? Unless, of course, you've seen Forgetting Sarah Marshall, in which case, it's probably a tad disturbing. Nevertheless, I do have a surprise for everyone. I spent a lot of time early last year trying to keep the buildbots green, especially the x64 Windows ones. This was quite enjoyable, at least initially.
Wow! :) I can barely find the words to express my feelings. You did an amazing job! Thank you very, *VERY* much!
Christian
Trent> Ten months, seven trips to MSU, six blown fuses and about
Trent> $60,000 later, I'm proud to introduce you all to Snakebite:
Trent> The Open Network! A network of around 37-ish servers of all
Trent> different shapes and sizes, spread over three sites,
Trent> specifically geared towards the needs of open source projects
Trent> like Python.Wow! A great resource no doubt.
I do notice that there is no Apple hardware or Mac OS X running on anything. Any chance of adding a XServe or something?
-- Skip Montanaro - skip@pobox.com - http://smontanaro.dyndns.org/
skip@pobox.com wrote:
Trent> Ten months, seven trips to MSU, six blown fuses and about Trent> $60,000 later, I'm proud to introduce you all to Snakebite: Trent> The Open Network! A network of around 37-ish servers of all Trent> different shapes and sizes, spread over three sites, Trent> specifically geared towards the needs of open source projects Trent> like Python.Wow! A great resource no doubt.
I do notice that there is no Apple hardware or Mac OS X running on anything. Any chance of adding a XServe or something?
Time we started making friends at Apple ...
regards Steve
Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jan 26, 2009, at 10:13 PM, Steve Holden wrote:
skip@pobox.com wrote:
Trent> Ten months, seven trips to MSU, six blown fuses and
about Trent> $60,000 later, I'm proud to introduce you all to
Snakebite: Trent> The Open Network! A network of around 37-ish servers
of all Trent> different shapes and sizes, spread over three sites, Trent> specifically geared towards the needs of open source
projects Trent> like Python.Wow! A great resource no doubt.
I do notice that there is no Apple hardware or Mac OS X running on anything. Any chance of adding a XServe or something?
Time we started making friends at Apple ...
Ed Moy was the Apple/Python contact I last spoke to. His email
address is:
emoy@apple.com
Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin)
iQCVAwUBSX8waHEjvBPtnXfVAQLtNQQApeBKE/VxEGxFWND4ozh77wRoSVL6WUdl ri+eGwGpzSNDsrNtpXu1AhtOK4L7cayJYvpAg1DoHcLX6Lg6S4lgdEGrhz1Qh09n +oLSbB5k5Nh+TK391rSQDUCUe2ZRkczJ8yYtf3Lfn9l91IlScJeIzsIIQqBU0CWT K1ugaZcN1oU= =s6Jg -----END PGP SIGNATURE-----
On Mon, Jan 26, 2009 at 10:13:56PM -0500, Steve Holden wrote:
skip@pobox.com wrote:
Trent> Ten months, seven trips to MSU, six blown fuses and about Trent> $60,000 later, I'm proud to introduce you all to Snakebite: Trent> The Open Network! A network of around 37-ish servers of all Trent> different shapes and sizes, spread over three sites, Trent> specifically geared towards the needs of open source projects Trent> like Python.Wow! A great resource no doubt.
I do notice that there is no Apple hardware or Mac OS X running on anything. Any chance of adding a XServe or something?
The only reason there isn't any Apple hardware yet is 'cause it's
expensive ;-) (In comparison to your average price for a server
off eBay, that is. I picked up six quad Opterons for $999 at one
point.)Time we started making friends at Apple ...
I'll give Jordan Hubbard a bell. He was the driving force behind
FreeBSD in the early days, but moved to Apple in early 2000. I
believe he's now the Director of Unix Technology now at Apple.
Actually, might as well just CC him now, no time like the present
;-)
Hi Jordan! I'll follow up in private and fill you in.
Trent.
A.M. Kuchling wrote:
On Mon, Jan 26, 2009 at 08:07:19PM -0600, skip@pobox.com wrote:
I do notice that there is no Apple hardware or Mac OS X running on anything. Any chance of adding a XServe or something?
Didn't Apple already give the PSF an XServe that's now at XS4ALL, but currently unused?
Worse than that: it is broken. It can't stay up for more than 30 seconds, before some watchdog mechanism reboots it - too short for me to find out what the problem is.
Regards, Martin
[SNIP: Trent announces his awesome secret project]
There is only one word I can say: wow! And to everyone who was involved, thank you!
This massive buildbot network is really going to help us toward making Python a top-class platform in term of portability. I am sure that having SSH access to the buildbots will encourage everyone to fix bugs quickly.
Finally, I have one question. Will these buildbots be open for general development work? It would be great to have access to them for creating new cross-platform libraries for Python.
-- Alexandre
On Mon, Jan 26, 2009 at 6:12 PM, Alexandre Vassalotti <alexandre@peadrop.com> wrote:
[SNIP: Trent announces his awesome secret project]
There is only one word I can say: wow! And to everyone who was involved, thank you!
This massive buildbot network is really going to help us toward making Python a top-class platform in term of portability. I am sure that having SSH access to the buildbots will encourage everyone to fix bugs quickly.
Finally, I have one question. Will these buildbots be open for general development work? It would be great to have access to them for creating new cross-platform libraries for Python.
Yep, I for example would love to use this resource to ensure that gmpy builds and runs well on more platforms than the usual mac/windows/linux set I have easy personal access to, and I suspect most other maintainers of third party Python add-ons feel similarly. I'm sure the line must be drawn somewhere, however, as there will never be enough computing resources in the "buildbot cloud" to support the thousands of Python add-ons that are not part of the Python distribution itself. A criterion such as "only 3rd party add ons whose maintainers are also Python committers" (even though it would favor gmpy) might be a tad arbitrary.
Alex
On Tue, Jan 27, 2009 at 08:20:55AM -0800, Alex Martelli wrote: -> On Mon, Jan 26, 2009 at 6:12 PM, Alexandre Vassalotti -> <alexandre@peadrop.com> wrote: -> > [SNIP: Trent announces his awesome secret project] -> > -> > There is only one word I can say: wow! And to everyone who was -> > involved, thank you! -> > -> > This massive buildbot network is really going to help us toward making -> > Python a top-class platform in term of portability. I am sure that -> > having SSH access to the buildbots will encourage everyone to fix bugs -> > quickly. -> > -> > Finally, I have one question. Will these buildbots be open for general -> > development work? It would be great to have access to them for -> > creating new cross-platform libraries for Python. -> -> Yep, I for example would love to use this resource to ensure that gmpy -> builds and runs well on more platforms than the usual -> mac/windows/linux set I have easy personal access to, and I suspect -> most other maintainers of third party Python add-ons feel similarly. -> I'm sure the line must be drawn somewhere, however, as there will -> never be enough computing resources in the "buildbot cloud" to support -> the thousands of Python add-ons that are not part of the Python -> distribution itself. A criterion such as "only 3rd party add ons -> whose maintainers are also Python committers" (even though it would -> favor gmpy) might be a tad arbitrary.
Hi all,
<delurk>
I don't expect any problem with people using this for whatever they want. If it becomes massively popular, well, then we'll just have to ramp up the resources...!
Right now, though, I'm more interested in recruiting "good citizens", people who can reasonably be expected to not kvetch too much as we develop out the infrastructure. I don't see any problem with extending the invitation to any long-term member of the Python community, and eventually anyone who writes a polite e-mail.
That having been said, I also don't have any problem with limiting access somewhat arbitrarily to people we "like". Gotta start somewhere!
On that front, incidentally, I hope to make the machines available to GSoC and GHOP students, and I will be involving MSU undergrads in their care and maintenance. I am also working to develop a more flexible and simple set of continuous build software; more on that at PyCon.
cheers, --titus
C. Titus Brown, ctb@msu.edu
C. Titus Brown wrote:
On Tue, Jan 27, 2009 at 08:20:55AM -0800, Alex Martelli wrote: -> On Mon, Jan 26, 2009 at 6:12 PM, Alexandre Vassalotti [...] -> > Finally, I have one question. Will these buildbots be open for general -> > development work? It would be great to have access to them for -> > creating new cross-platform libraries for Python. -> -> Yep, I for example would love to use this resource to ensure that gmpy [understandable special pleading]
Hi all,
<delurk>
I don't expect any problem with people using this for whatever they want. If it becomes massively popular, well, then we'll just have to ramp up the resources...!
That's the spirit! Demand is evidence of the need to supply!
Right now, though, I'm more interested in recruiting "good citizens", people who can reasonably be expected to not kvetch too much as we develop out the infrastructure. I don't see any problem with extending the invitation to any long-term member of the Python community, and eventually anyone who writes a polite e-mail.
So that will eliminate about 25% of the community right there ;-)
That having been said, I also don't have any problem with limiting access somewhat arbitrarily to people we "like". Gotta start somewhere!
On that front, incidentally, I hope to make the machines available to GSoC and GHOP students, and I will be involving MSU undergrads in their care and maintenance. I am also working to develop a more flexible and simple set of continuous build software; more on that at PyCon.
I'm looking forward to that.
regards Steve
Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/
Trent Nelson wrote:
Sun, Google and Canonical have also expressed a lot of interest in the project -- I stopped asking for hardware a while back though as we've literally run out of space to host it all. The website is live, but the content is a bit sparse at the moment, excluding the poorly worded front page and the reasonably accurate network page: http://www.snakebite.org http://www.snakebite.org/network It'll probably be a few weeks before you can start logging in and doing stuff. The HPCC/CSE server room at MSU is about to have walls knocked in and ramps built in order to accommodate a giant PDU that has been sitting outside it for about six months; the Snakebite rack is going to get shuffled around a bit so I figure there's not much point going live before that's taken care of. Other than that, I'm just happy to get this off my chest, ten months is a freakin' long time to try and keep something like this a secret ;-)
Great stuff Trent! I was wondering how you were doing.
I really appreciate what it takes to put these open resources together ;) There's a lot of moving parts :)
Cheers, Jim
BTW.
We now have zone servers in the OpenSolaris test farm, and I plan to add guest os servers in the next few weeks using ldoms (sparc) and xvm (x64). The zone servers provide whole root zones, which should be a good development environment for most projects. Check it out:
http://test.opensolaris.org/testfarm http://www.opensolaris.org/os/community/testing/testfarm/zones/
Let me know if there is interest from the python community to manage one of the test farm servers for python development. Besides the general use machines, the php community is already managing a T2000 server.
Trent Nelson schrieb:
And last but not least, thanks to Guido and all the Python committers for their tireless efforts to date. Although the sheer elegance of the language is what initially attracted me to Python, it was the developer community that made me want to stick around. Snakebite is my gift to you!
And what a gift it is! Add me to the list of speechless congratulators.
Georg
On Mon, Jan 26, 2009 at 11:32 PM, Trent Nelson <python-committers-list@trentnelson.com> wrote:
Nevertheless, I do have a surprise for everyone.
A surprise, indeed! Golly! I can already think of several ways to use such a fantastic resource. Some of them might even be legal. :-)
Thank you, Trent and all involved.
Just one question: who is Sarah Marshall, and is it safe to Google for her while at work?
Mark
It's an R rated romantic comedy from the Apatow team, so I guess that depends on your employer ;-)
</F>
On Tue, Jan 27, 2009 at 11:38 AM, Mark Dickinson <dickinsm@gmail.com> wrote:
On Mon, Jan 26, 2009 at 11:32 PM, Trent Nelson <python-committers-list@trentnelson.com> wrote:
Nevertheless, I do have a surprise for everyone.
A surprise, indeed! Golly! I can already think of several ways to use such a fantastic resource. Some of them might even be legal. :-)
Thank you, Trent and all involved.
Just one question: who is Sarah Marshall, and is it safe to Google for her while at work?
Mark
python-committers mailing list python-committers@python.org http://mail.python.org/mailman/listinfo/python-committers
On Mon, Jan 26, 2009 at 9:32 PM, Trent Nelson <python-committers-list@trentnelson.com> wrote:
. .
Ten months, seven trips to MSU, six blown fuses and about $60,000 later, I'm proud to introduce you all to Snakebite: The Open Network! A network of around 37-ish servers of all different shapes and sizes, spread over three sites, specifically geared towards the needs of open source projects like Python.
Every CPython, Jython, IronPython and PyPy committer will have access to every development server on the network. I've also extended the offer to prominent Python projects like Django and Twisted.
Eventually, I'll invite other open source projects to participate (Apache, Subversion, MySQL, Postgres, etc), but the network is my gift to All Things Python, first and foremost, so Python projects will always get preferential treatment.
Support for the initiative so far has been nothing short of sublime.
Microsoft jumped on board and provided unlimited MSDN licenses in less time than it took me to write them an e-mail asking for stuff.
I sent HP an e-mail asking if they could spare a Tru64 license, and maybe 2GB of RAM for an extremely crappy Itanium box I bought off eBay. They saw my Tru64 license request and raised with media and licenses to the latest version of HP-UX.
Unfortunately, it was too much trouble for them to try and source 2GB of RAM for the Itanium I bought. So, instead, they shipped two massive quad Itanium 2 RX-5670s, chock full of 73GB 15k disks and no less than 78GB of RAM between the two servers; 32GB in one and 46GB in the other. Well then.
(I'd hate to think what would have turned up had I asked them for two quad Itanium monsters.)
Sun, Google and Canonical have also expressed a lot of interest in the project -- I stopped asking for hardware a while back though as we've literally run out of space to host it all.
The website is live, but the content is a bit sparse at the moment, excluding the poorly worded front page and the reasonably accurate network page: http://www.snakebite.org http://www.snakebite.org/network
It'll probably be a few weeks before you can start logging in and doing stuff. The HPCC/CSE server room at MSU is about to have walls knocked in and ramps built in order to accommodate a giant PDU that has been sitting outside it for about six months; the Snakebite rack is going to get shuffled around a bit so I figure there's not much point going live before that's taken care of.
Other than that, I'm just happy to get this off my chest, ten months is a freakin' long time to try and keep something like this a secret ;-)
. .
Very very nice Trent :)
I remember when you raised the idea last year and it seemed awesome, but it turned out to be better than awesome! It is just incredible how people like you are able to donate so much time and effort to open source projects.
Looks like the season for snake mites, six-spotted tiger beetle, and what not, is now open.
Regards,
-- -- Guilherme H. Polo Goncalves
Congratulations, Trent. Thank you for your leadership and giving us the chance to participate. This project has captured the imagination of the Open Source Technology Center at Microsoft and we look forward to doing more to support the project in the future.
Best regards,
Sam
Sam Ramji | Sr. Director, Platform Strategy | Microsoft Corporation | +1 510 913 6495 | sramji@microsoft.com
-----Original Message----- From: Trent Nelson [mailto:python-committers-list@trentnelson.com] Sent: Monday, January 26, 2009 3:33 PM To: python-committers@python.org Cc: bob.gobeille@hp.com; titus@idyll.org; glynn.foster@sun.com; jim.walker@sun.com; ted.leung@sun.com; Frank.Wierzbicki@Sun.COM; Hank Janssen; Garrett Serack; punch@cse.msu.edu; Sam Ramji; Chip Norkus; Dino Viehland; Jimmy Schementi; Dave Fugate; stockman@cse.msu.edu; pitcher2@cse.msu.edu; tnelson@onresolve.com; mark@canonical.com; exarkun@divmod.com; gylph@divmod.com; lhawthorn@google.com; doko@ubuntu.com; fuzzyman@voidspace.org.uk; oubiwann@divmod.com; jacob.kaplanmoss@gmail.com; psf@python.org; climber@cse.msu.edu Subject: I've got a surprise for you!
Catchy subject eh? Unless, of course, you've seen Forgetting Sarah
Marshall, in which case, it's probably a tad disturbing.
Nevertheless, I do have a surprise for everyone. I spent a lot of
time early last year trying to keep the buildbots green, especially
the x64 Windows ones. This was quite enjoyable, at least initially.
By the time I got back to London after PyCon last year, the buildbot
honeymoon was wearing off. As there's usually only one person that
has access to a given buildbot, and that person is rarely you, it
can be a right pain in the ass trying to debug problems you can't
reproduce on platforms you don't have access to (especially if all
you've got is an error message that doesn't make sense).
I was convinced there must be a better way.
Around mid-April, the buildbot on my FreeBSD 6.x box I had recently
set up kept failing on on setitimer tests. I received an email from
a chap named Guilherme Polo who had seen the buildbot test failures
and wanted to assist. I was swamped with client work at the time; I
wasn't able to run any of the test scripts he sent me, but, hey, he
seemed like a nice chap, so I thought what the hell, and just gave
him an account on the box and checked out a copy of trunk in his
home directory.
Now, mind you, this was before Guilherme was a committer; I didn't
know the guy from a bar of soap. For all I knew, he could have been
using the shell account to launch a massive DDoS or phishing scam.
However, no more than five minutes after I created his account, he'd
diagnosed the problem, replicated the behaviour in a C program, and,
with a bit of googling, figured out what was wrong and proposed a
fix.
And that's when it hit me. Buildbots are fine when everything is
running smoothly, but nothing compares to actually having access to
a system when you're trying to debug something.
So, I thought to myself, why not buy a couple of clunky old boxes
off eBay and donate them to the PSF, such that all developers had
access to them? I dropped a note to Guido and Neal, they put me
in touch with Titus, who had just accepted a position at Michigan
State, and, well...
Ten months, seven trips to MSU, six blown fuses and about $60,000
later, I'm proud to introduce you all to Snakebite: The Open Network!
A network of around 37-ish servers of all different shapes and sizes,
spread over three sites, specifically geared towards the needs of
open source projects like Python.
Every CPython, Jython, IronPython and PyPy committer will have access
to every development server on the network. I've also extended the
offer to prominent Python projects like Django and Twisted.
Eventually, I'll invite other open source projects to participate
(Apache, Subversion, MySQL, Postgres, etc), but the network is my
gift to All Things Python, first and foremost, so Python projects
will always get preferential treatment.
Support for the initiative so far has been nothing short of sublime.
Microsoft jumped on board and provided unlimited MSDN licenses in
less time than it took me to write them an e-mail asking for stuff.
I sent HP an e-mail asking if they could spare a Tru64 license, and
maybe 2GB of RAM for an extremely crappy Itanium box I bought off
eBay. They saw my Tru64 license request and raised with media and
licenses to the latest version of HP-UX.
Unfortunately, it was too much trouble for them to try and source
2GB of RAM for the Itanium I bought. So, instead, they shipped two
massive quad Itanium 2 RX-5670s, chock full of 73GB 15k disks and
no less than 78GB of RAM between the two servers; 32GB in one and
46GB in the other. Well then.
(I'd hate to think what would have turned up had I asked them for
two quad Itanium monsters.)
Sun, Google and Canonical have also expressed a lot of interest in
the project -- I stopped asking for hardware a while back though as
we've literally run out of space to host it all.
The website is live, but the content is a bit sparse at the moment,
excluding the poorly worded front page and the reasonably accurate
network page:
http://www.snakebite.org
http://www.snakebite.org/network
It'll probably be a few weeks before you can start logging in and
doing stuff. The HPCC/CSE server room at MSU is about to have walls
knocked in and ramps built in order to accommodate a giant PDU that
has been sitting outside it for about six months; the Snakebite rack
is going to get shuffled around a bit so I figure there's not much
point going live before that's taken care of.
Other than that, I'm just happy to get this off my chest, ten months
is a freakin' long time to try and keep something like this a secret
;-)
Thanks to everyone who has contributed so far. Titus, Bill, Adam,
Kelly, George: Snakebite wouldn't be anything more than a twinkle
in my eye if it weren't for the support MSU has thrown behind the
project, thank you for all your efforts to date. Hank, Garrett and
Sam: having the support of Microsoft from very early on has been a
huge boost and the MSDN licenses have already been invaluable. Bob,
well, what can I say, there was a period there where every e-mail
thread between us seemed to result in something being shipped from
HP to MSU; thanks to you and HP's open source labs.
And last but not least, thanks to Guido and all the Python committers
for their tireless efforts to date. Although the sheer elegance of
the language is what initially attracted me to Python, it was the
developer community that made me want to stick around.
Snakebite is my gift to you!
Trent.
I've just set up a mailing list for those that want to carry on with
discussions; this CC list is getting a bit unwieldy. Subscription
URL: http://groups.google.com/group/snakebite-list. E-mail address
is snakebite-list@googlegroups.com.
I'll be sending the rest of my replies there after this e-mail.
Trent.
On 2009-01-27 21:01, Trent Nelson wrote:
I've just set up a mailing list for those that want to carry on with discussions; this CC list is getting a bit unwieldy. Subscription URL: http://groups.google.com/group/snakebite-list. E-mail address is snakebite-list@googlegroups.com. I'll be sending the rest of my replies there after this e-mail.
Would it be possible to have such a mailing list setup on python.org ? Or perhaps have Mailman running on snakebite.org ?
Thanks,
Marc-Andre Lemburg eGenix.com
Professional Python Services directly from the Source (#1, Jan 28 2009)
Python/Zope Consulting and Support ... http://www.egenix.com/ mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
::: Try our new mxODBC.Connect Python Database Interface for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Jan 28, 2009, at 11:51 AM, M.-A. Lemburg wrote:
On 2009-01-27 21:01, Trent Nelson wrote:
I've just set up a mailing list for those that want to carry on
with discussions; this CC list is getting a bit unwieldy. Subscription URL: http://groups.google.com/group/snakebite-list. E-mail address is snakebite-list@googlegroups.com.I'll be sending the rest of my replies there after this e-mail.
Would it be possible to have such a mailing list setup on python.org ? Or perhaps have Mailman running on snakebite.org ?
+1 for either. Send a message to postmaster@python.org if you want
the former. It should definitely be approved.
Barry
-----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (Darwin)
iQCVAwUBSYCkAXEjvBPtnXfVAQLlxgP9F/14phSrD+HXsOLcgaBaymORuILxFpFp AXVH7o1tebi7oOXamB/2dMJaUkVAz7Sf8rtnv4tP0l/Z/1kHJFZrRZQaLTS3KxYh 6ooBVHkzokPGtmOS5Ms8ijuMiI3e3OBHL90/gkVf3O3G1+ELM3CTeqp5HgHD6/J/ DY7nYkvJfS4= =6wKo -----END PGP SIGNATURE-----
On Wed, Jan 28, 2009 at 01:29:21PM -0500, Barry Warsaw wrote:
On Jan 28, 2009, at 11:51 AM, M.-A. Lemburg wrote:
On 2009-01-27 21:01, Trent Nelson wrote:
I've just set up a mailing list for those that want to carry on
with discussions; this CC list is getting a bit unwieldy. Subscription URL: http://groups.google.com/group/snakebite-list. E-mail address is snakebite-list@googlegroups.com.I'll be sending the rest of my replies there after this e-mail.
Would it be possible to have such a mailing list setup on python.org ? Or perhaps have Mailman running on snakebite.org ?
+1 for either. Send a message to postmaster@python.org if you want
the former. It should definitely be approved.
I'll follow up with the interested parties in private.
Trent.
participants (22)
-
"Martin v. Löwis" -
A.M. Kuchling -
Alex Martelli -
Alexandre Vassalotti -
Barry Warsaw -
Brett Cannon -
C. Titus Brown -
Christian Heimes -
Frank Wierzbicki -
Fredrik Lundh -
Georg Brandl -
Guilherme Polo -
Hank Janssen -
Jesse Noller -
Jim Walker -
M.-A. Lemburg -
Mark Dickinson -
Sam Ramji -
skip＠pobox.com -
Steve Holden -
Trent Nelson -
Trent Nelson