Stable build slaves authority
I have appointed Antoine Pitrou as the authority/manager for which build slave are considered stable. If you want to get a certain slave elevated or demoted, you have to convince him. I would also like to ask release managers to take the stable list serious: test failures in a stable slave should be considered as release blockers (demoting a slave to unstable may be a resolution, though). Of course, release managers can deliberately chose to ignore specific blockers, anyway. Regards, Martin
On Wed, 13 Oct 2010 22:55:21 +0200 "Martin v. Löwis" <martin@v.loewis.de> wrote:
I have appointed Antoine Pitrou as the authority/manager for which build slave are considered stable. If you want to get a certain slave elevated or demoted, you have to convince him.
Thank you Martin! I've added a couple of build slaves to the stable list: - x86 Windows7 (from David Bolen) - x86 Ubuntu (8.04.3 LTS, from David Bolen) - AMD64 Gentoo (from Dirkjan Ochtman) - x86 Snow Leopard (from Stephen Hansen) This is in hope that the stable list represents a greater diversity of systems and architectures. If this turns out to be too ambitious, I'll have to shrink the stable list again. (you'll notice that we have currently no 64-bit Windows machine although 64-bit support under Windows has specific issues) I would like to stress that help from the buildbot owners is welcome when some platform-specific issues need investigating and solving. Being proactive is even better! Regards Antoine.
On 10/13/10 2:47 PM, Antoine Pitrou wrote:
(you'll notice that we have currently no 64-bit Windows machine although 64-bit support under Windows has specific issues)
Provided its not a problem that its a VM, I have a hefty 64-bit Win7 Professional instance that I can put a buildslave on. Despite being a VM it gets ownership of two cores and 4 gigs of RAM, so should be plenty fast to handle the load. And I do run it 24/7. -- Stephen Hansen ... Also: Ixokai ... Mail: me+python (AT) ixokai (DOT) io ... Blog: http://meh.ixokai.io/
Am 14.10.2010 00:08, schrieb Stephen Hansen:
On 10/13/10 2:47 PM, Antoine Pitrou wrote:
(you'll notice that we have currently no 64-bit Windows machine although 64-bit support under Windows has specific issues)
Provided its not a problem that its a VM, I have a hefty 64-bit Win7 Professional instance that I can put a buildslave on. Despite being a VM it gets ownership of two cores and 4 gigs of RAM, so should be plenty fast to handle the load. And I do run it 24/7.
So far, we didn't have problems with VMs. Please be aware that Windows poses its own challenges. Often, builds or testsuite runs end up with popup windows, which then hang subsequent builds. You often get dozens of them to click away. So operating a Windows slave is much more tedious than a Unix one. Regards, Martin
On 10/13/10 3:14 PM, "Martin v. Löwis" wrote:
Am 14.10.2010 00:08, schrieb Stephen Hansen:
On 10/13/10 2:47 PM, Antoine Pitrou wrote:
(you'll notice that we have currently no 64-bit Windows machine although 64-bit support under Windows has specific issues)
Provided its not a problem that its a VM, I have a hefty 64-bit Win7 Professional instance that I can put a buildslave on. Despite being a VM it gets ownership of two cores and 4 gigs of RAM, so should be plenty fast to handle the load. And I do run it 24/7.
So far, we didn't have problems with VMs.
Please be aware that Windows poses its own challenges. Often, builds or testsuite runs end up with popup windows, which then hang subsequent builds. You often get dozens of them to click away. So operating a Windows slave is much more tedious than a Unix one.
Windows always poses its own challenges. :) That's why I have the VM (and three others for older versions of windows, that just aren't on all the time like that one is) to begin with*, for testing out my day-job work. I'll give it a go; I have all the software needed to run the buildbot on it already besides VC Express, which I'm installing now. If ultimately it becomes too much of a pain, I'll go back to just providing the mac. But, I actually have a vested interest in upgrading our Python to 64-bit in the next few months, so! I'm motivated. I'll let you know when I have everything installed so you can add a buildslave account. -- Stephen Hansen ... Also: Ixokai ... Mail: me+python (AT) ixokai (DOT) io ... Blog: http://meh.ixokai.io/
I'll give it a go; I have all the software needed to run the buildbot on it already besides VC Express, which I'm installing now. If ultimately it becomes too much of a pain, I'll go back to just providing the mac. But, I actually have a vested interest in upgrading our Python to 64-bit in the next few months, so! I'm motivated.
That won't work, will it? VC Express doesn't come with an AMD64 compiler (I *think* it's possible to use the SDK one, but this again is more complicated). Regards, Martin
On 10/13/10 3:42 PM, "Martin v. Löwis" wrote:
I'll give it a go; I have all the software needed to run the buildbot on it already besides VC Express, which I'm installing now. If ultimately it becomes too much of a pain, I'll go back to just providing the mac. But, I actually have a vested interest in upgrading our Python to 64-bit in the next few months, so! I'm motivated.
That won't work, will it? VC Express doesn't come with an AMD64 compiler (I *think* it's possible to use the SDK one, but this again is more complicated).
Oh! Well if it takes a paid version of VS, then I won't be able to do it. I'll experiment with getting the SDK and using that and seeing if I can make it work. -- Stephen Hansen ... Also: Ixokai ... Mail: me+python (AT) ixokai (DOT) io ... Blog: http://meh.ixokai.io/
On Wed, Oct 13, 2010 at 17:42, "Martin v. Löwis" <martin@v.loewis.de> wrote:
I'll give it a go; I have all the software needed to run the buildbot on it already besides VC Express, which I'm installing now. If ultimately it becomes too much of a pain, I'll go back to just providing the mac. But, I actually have a vested interest in upgrading our Python to 64-bit in the next few months, so! I'm motivated.
That won't work, will it? VC Express doesn't come with an AMD64 compiler (I *think* it's possible to use the SDK one, but this again is more complicated).
Regards, Martin
Correct. There are a few hacky ways to get Express to use the x64 SDK, or so I read. I have a Server 2008 R2 x64 box with the full Visual Studio that I could add to the buildbot fleet. It's a dual core with 4 GB of RAM, plenty of disk space, and it runs 24/7. I'll see about getting that setup as a build slave.
-On [20101014 00:55], Brian Curtin (brian.curtin@gmail.com) wrote:
Correct. There are a few hacky ways to get Express to use the x64 SDK, or so I read.
I think Martin meant that you wouldn't need VS Express if you install the Windows SDK, since it provides all the tools in the SDK to build Python. Sorry Martin, haven't had the time at work to test the latest trunks out with just the SDK. I'll see if I can free some time for that in the coming days. -- Jeroen Ruigrok van der Werven <asmodai(-at-)in-nomine.org> / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェン http://www.in-nomine.org/ | http://www.rangaku.org/ | GPG: 2EAC625B Seize from every moment its unique novelty and do not prepare your joys...
On 10/13/10 10:28 PM, Jeroen Ruigrok van der Werven wrote:
-On [20101014 00:55], Brian Curtin (brian.curtin@gmail.com) wrote:
Correct. There are a few hacky ways to get Express to use the x64 SDK, or so I read.
I think Martin meant that you wouldn't need VS Express if you install the Windows SDK, since it provides all the tools in the SDK to build Python.
There's mixed signals here, and I'm not sure what they all mean. I have a Win7-64bit box that I am willing to use to run a buildslave, if its possible to do so. #python-dev thought that VS express was all that was needed; then here, it seemed to me that Martin said that you needed the full version of VS or perhaps a complex setup with the SDK compiler; but you seem to be interpreting Martin that the SDK provides everything and nothing else is needed. Then again on top of that, my offer may be mooted-- if Brian Curtin is going to host a x86_64 windows slave then I don't need to worry about this because its being provided otherwise. I'm willing to put up with the particular windows-specific difficulties that go with running a buildslave (especially with David Bolen's AutoIt scripts which may ease things): but I'm not entirely sure from these varied results if its even possible or needed. So, my questions are: 1. Is someone else (Hi, Brian) providing a 64-bit windows slave, so there isn't actually any need for me to go through the effort of it? 2. If not, is all that's needed is the SDK to build 64-bit Python? 3. Or, does one have to use a combination of VS-Express + the SDK in a "hack"y way (as some seem to claim, but this last mail seems to indicate otherwise) to get it done? Basically, it comes down to: 'it' being a 64-bit windows slave, is it actually needed from me (i.e., is a more apt expert not providing it), and can anyone actually say what the requirements are for making it happen? At the moment I'm uncertain if its even needed or worthwhile to go through the effort to get the whole visual studio environment set up. I have computing resources, cycles, and time that's free to offer up: but the differing responses here makes me unsure if I'm being useful or not in trying here :) -- Stephen Hansen ... Also: Ixokai ... Mail: me+python (AT) ixokai (DOT) io ... Blog: http://meh.ixokai.io/
#python-dev thought that VS express was all that was needed; then here, it seemed to me that Martin said that you needed the full version of VS or perhaps a complex setup with the SDK compiler; but you seem to be interpreting Martin that the SDK provides everything and nothing else is needed.
I think I forgot earlier discussion on how to use the SDK. It may well be that it can be used to build Python with no VS installed, but it's not what I meant - I meant that I actually don't know.
1. Is someone else (Hi, Brian) providing a 64-bit windows slave, so there isn't actually any need for me to go through the effort of it?
With the Windows build slaves: they come and go. So having two of them might be a good idea.
2. If not, is all that's needed is the SDK to build 64-bit Python?
I think that you will need to find out on your own. And, even if it's possible, the next question is whether what the buildbot scripts do actually works without VS (but we could adjust them).
Basically, it comes down to: 'it' being a 64-bit windows slave, is it actually needed from me (i.e., is a more apt expert not providing it), and can anyone actually say what the requirements are for making it happen?
As for "needed", well, we a are all volunteers. This project will certainly survive without a 64-bit Windows build slave (it did so in the last months). As for saying what the requirements are: probably nobody can.
I have computing resources, cycles, and time that's free to offer up: but the differing responses here makes me unsure if I'm being useful or not in trying here :)
Well, the differing responses really demonstrate that it's also expertise that is lacking throughout. If you are motivated, please try to find out how to build Python on AMD64 with free-as-in-beer Microsoft tools, and provide a patch to PCbuild/readme.txt. Having such a setup then tested regularly is certainly worthwhile. Regards, Martin
On Thu, Oct 14, 2010 at 01:00, Stephen Hansen <me+python@ixokai.io<me%2Bpython@ixokai.io>
wrote:
On 10/13/10 10:28 PM, Jeroen Ruigrok van der Werven wrote:
-On [20101014 00:55], Brian Curtin (brian.curtin@gmail.com) wrote:
Correct. There are a few hacky ways to get Express to use the x64 SDK, or so I read.
I think Martin meant that you wouldn't need VS Express if you install the Windows SDK, since it provides all the tools in the SDK to build Python.
There's mixed signals here, and I'm not sure what they all mean. I have a Win7-64bit box that I am willing to use to run a buildslave, if its possible to do so.
#python-dev thought that VS express was all that was needed; then here, it seemed to me that Martin said that you needed the full version of VS or perhaps a complex setup with the SDK compiler; but you seem to be interpreting Martin that the SDK provides everything and nothing else is needed.
Then again on top of that, my offer may be mooted-- if Brian Curtin is going to host a x86_64 windows slave then I don't need to worry about this because its being provided otherwise.
I'm willing to put up with the particular windows-specific difficulties that go with running a buildslave (especially with David Bolen's AutoIt scripts which may ease things): but I'm not entirely sure from these varied results if its even possible or needed.
So, my questions are: 1. Is someone else (Hi, Brian) providing a 64-bit windows slave, so there isn't actually any need for me to go through the effort of it?
I'm planning to -- just need a little time to sit down and get the box ready to go. I would say that one build slave is good, two is better. If you feel up to the task, go right ahead. As one of the few Windows users around here, I'd certainly appreciate more Windows testing. One of my motivations in providing a build slave is execution of the newly added os.symlink support starting in Windows Vista. Since os.symlink requires a certain privilege, it's not tested on any of the current build slaves. I plan to work out a way for my build slave to get these tests executed, rather than just running them on my desktop boxes.
2. If not, is all that's needed is the SDK to build 64-bit Python?
Sorry for my confusion earlier - I was focused on the VS Express part, rather than the minimum requirement to build Python (which doesn't *require* VS at all, it's just a nice tool to build it). Supposedly the 64-bit SDK by itself should build a 64-bit Python -- I say "supposedly" because I haven't done this myself. I've read that only using the 32-bit SDK does this fine -- also have not done this myself. I understand that doesn't really answer your question, but hopefully my previous involvement seems more clear.
On Wed, 13 Oct 2010 17:50:49 -0500 Brian Curtin <brian.curtin@gmail.com> wrote: [...]
I have a Server 2008 R2 x64 box with the full Visual Studio that I could add to the buildbot fleet. It's a dual core with 4 GB of RAM, plenty of disk space, and it runs 24/7.
The adventures of http://bugs.python.org/issue9778 show we would really benefit from a 64-bit Windows build slave. Regards Antoine.
"Martin v. Löwis" <martin@v.loewis.de> writes:
Please be aware that Windows poses its own challenges. Often, builds or testsuite runs end up with popup windows, which then hang subsequent builds. You often get dozens of them to click away. So operating a Windows slave is much more tedious than a Unix one.
For anyone else considering a Windows buildbot, recently I started using an AutoIt script in the background to automatically clear any RTL dialogs (which seems to come and go as an issue but got really bad recently in one of the branches, and I lost track of when Python itself stopped disabling them during tests), and it's been working well so far. I also have a tweak in the local buildbot code to disable all Windows-based (non-RTL) dialogs during test runs. I'd be happy to provide both bits to anyone else starting a Windows buildbot. There's still a regular problem of stranded python_d processes in the background (something shared with my OSX tiger slave, but there I can run a script to detect processes owned by init and kill them). So periodic manual checks/cleanup is still definitely needed. -- David
On Wed, 13 Oct 2010 15:08:24 -0700 Stephen Hansen <me+python@ixokai.io> wrote:
On 10/13/10 2:47 PM, Antoine Pitrou wrote:
(you'll notice that we have currently no 64-bit Windows machine although 64-bit support under Windows has specific issues)
Provided its not a problem that its a VM, I have a hefty 64-bit Win7 Professional instance that I can put a buildslave on. Despite being a VM it gets ownership of two cores and 4 gigs of RAM, so should be plenty fast to handle the load. And I do run it 24/7.
Sounds good. Now of course you have to setup a buildslave on it, and ask Martin to register the slave on the master (unless I'm supposed to do it myself now?). Regards Antoine.
Am 13.10.2010 22:55, schrieb "Martin v. Löwis":
I have appointed Antoine Pitrou as the authority/manager for which build slave are considered stable. If you want to get a certain slave elevated or demoted, you have to convince him.
I would also like to ask release managers to take the stable list serious: test failures in a stable slave should be considered as release blockers (demoting a slave to unstable may be a resolution, though). Of course, release managers can deliberately chose to ignore specific blockers, anyway.
Very nice. http://www.python.org/dev/buildbot/stable/ is completely green at the moment -- which means that I can now indeed take failures seriously in the future. Previously, two of four "stables" for py3k were not even connected for ages (and yes, I should have complained earlier. As we progress into the beta stage, I will take buildbot failures and release blockers more seriously). cheers, Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
On Thu, 14 Oct 2010 08:07:46 +0200 Georg Brandl <g.brandl@gmx.net> wrote:
Very nice. http://www.python.org/dev/buildbot/stable/ is completely green at the moment -- which means that I can now indeed take failures seriously in the future. Previously, two of four "stables" for py3k were not even connected for ages (and yes, I should have complained earlier. As we progress into the beta stage, I will take buildbot failures and release blockers more seriously).
For the record, I've setup one of the stable buildbots ("AMD64 Gentoo Wide") to test wide unicode buils. Also, one of the unstable buildbots ("x86 debian parallel") now runs its tests with -OO, at Georg's request. Regards Antoine.
Am 14.10.2010 18:55, schrieb Antoine Pitrou:
On Thu, 14 Oct 2010 08:07:46 +0200 Georg Brandl <g.brandl@gmx.net> wrote:
Very nice. http://www.python.org/dev/buildbot/stable/ is completely green at the moment -- which means that I can now indeed take failures seriously in the future. Previously, two of four "stables" for py3k were not even connected for ages (and yes, I should have complained earlier. As we progress into the beta stage, I will take buildbot failures and release blockers more seriously).
For the record, I've setup one of the stable buildbots ("AMD64 Gentoo Wide") to test wide unicode buils. Also, one of the unstable buildbots ("x86 debian parallel") now runs its tests with -OO, at Georg's request.
Excellent, thanks! Georg -- Thus spake the Lord: Thou shalt indent with four spaces. No more, no less. Four shall be the number of spaces thou shalt indent, and the number of thy indenting shall be four. Eight shalt thou not indent, nor either indent thou two, excepting that thou then proceed to four. Tabs are right out.
participants (7)
-
"Martin v. Löwis" -
Antoine Pitrou -
Brian Curtin -
David Bolen -
Georg Brandl -
Jeroen Ruigrok van der Werven -
Stephen Hansen