On Tue, Sep 2, 2014 at 5:53 PM, Nick Coghlan wrote:

On 3 Sep 2014 08:15, "Victor Stinner" wrote:

...

x86 RHEL 6 3.x: TestReadline.test_init() fails, issue #19884. I don't have to this platform, I don't know how to fix it.

Sorry, I haven't been a very good maintainer for that buildbot (the main reason it never graduated to the "stable" list). If you send me your public SSH key, I can add it (I think - if not, I can ask Luke to do it). Alternatively, CentOS 6 may exhibit the same problem.

From a completely different perspective, does anyone have experience with using BuildBot with OpenStack hosted clients? We may be able to take advantage of the PSF's new(ish) Rackspace infrastructure to provide more stable test VMs.

Is this a Buildbot feature (as in Buildbot master spins up VMs fresh for a test run, or something), or do you just want to spin up a bunch of VMs, give access, and we configure them the same as we do today?

On Wed, 3 Sep 2014 08:53:51 +1000 Nick Coghlan wrote:

On 3 Sep 2014 08:15, "Victor Stinner" wrote:

...

x86 RHEL 6 3.x: TestReadline.test_init() fails, issue #19884. I don't have to this platform, I don't know how to fix it.

Sorry, I haven't been a very good maintainer for that buildbot (the main reason it never graduated to the "stable" list). If you send me your public SSH key, I can add it (I think - if not, I can ask Luke to do it). Alternatively, CentOS 6 may exhibit the same problem.

From a completely different perspective, does anyone have experience with using BuildBot with OpenStack hosted clients? We may be able to take advantage of the PSF's new(ish) Rackspace infrastructure to provide more stable test VMs.

I'm not sure that's an answer to the problem. What we need is not more machines, but dedicated buildbot maintainers. That also means being willing to diagnose issues themselves rather than kindly offer SSH access, by the way ;-) (I can't speak for Victor, but being offered SSH access when I point out an issue on a remote machine is really a depressing answer to get, as far as I'm concerned) And, we also need some people to look after the master configuration and status - yes, that would have been me, but I've not been very motivated recently :-) Regards Antoine.

On Sat, Oct 11, 2014 at 2:45 AM, Jesus Cea wrote:

On 03/09/14 02:37, Antoine Pitrou wrote:

...

I'm not sure that's an answer to the problem. What we need is not more machines, but dedicated buildbot maintainers.

I would love to get an email if my buildbots are consistently RED for a few hours.

In the past Antoine, Victor and others pinged me about issues in my buildbots (OpenIndiana). I feel shame and I think it is not very productive either.

Likewise. In the earliest days of my buildbots, I kept a very close eye on them (which was good; they had some occasional network glitches, which I had to resolve), but once they became stable, I stopped watching. Now, I almost never check them - as long as the VM's running, I basically assume all's well. It'd be very helpful if a computer could auto-email any time they're down for any length of time. Is the info available in some way? Could I write a little monitor at my end that asks every hour if my buildbots can be seen? ChrisA

On Fri, 10 Oct 2014 18:00:06 +0200, Jesus Cea wrote:

On 10/10/14 17:56, Chris Angelico wrote:

...

Could I write a little monitor at my end that asks every hour if my buildbots can be seen?

AFAIK maintainers already get an email if the buildbot vanishes long enough. I am more interested in getting an email when my buildbot is consistently red because somebody committed something it doesn't like two months ago...

I think they are supposed to, but I've never gotten one, not even when my gentoo buildbots suffered a hardware failure. A month or so later Antoine emailed me, and I told him to remove them at least for now, since I don't currently have replacement hardware. (I'm hoping to fix that, but I have to find the time...) That said, there has never as far as I know been an active hook to email the maintainer when the buildbot is red. I don't know if buildbot supports that or not, so that would be the first thing to check. --David

On Fri, 10 Oct 2014 12:08:54 -0400 "R. David Murray" wrote:

On Fri, 10 Oct 2014 18:00:06 +0200, Jesus Cea wrote:

...

On 10/10/14 17:56, Chris Angelico wrote:

...

Could I write a little monitor at my end that asks every hour if my buildbots can be seen?

AFAIK maintainers already get an email if the buildbot vanishes long enough. I am more interested in getting an email when my buildbot is consistently red because somebody committed something it doesn't like two months ago...

I think they are supposed to, but I've never gotten one, not even when my gentoo buildbots suffered a hardware failure.

As far as I remember it's a relaying problem on the SMTP server. Perhaps Benjamin can confirm.

That said, there has never as far as I know been an active hook to email the maintainer when the buildbot is red. I don't know if buildbot supports that or not, so that would be the first thing to check.

I don't know if it would support e-mailing the maintainer rather than whoever committed the last changes. Besides, I'm afraid our test suite isn't stable enough for such e-mail notifications to be bearable... Regards Antoine.

Nick Coghlan writes:

Sorry, I haven't been a very good maintainer for that buildbot (the main reason it never graduated to the "stable" list). If you send me your public SSH key, I can add it (I think - if not, I can ask Luke to do it). Alternatively, CentOS 6 may exhibit the same problem.

I wonder how many of these buildbots could be maintained by the kind of folks who show up on core-mentorship asking "how can I help?" Just a thought -- I wouldn't be surprised if the reaction is universal horror and the answer is "Are you crazy? Zero! Z-E-R-O!!" And of course most want to write code, not sysadm.

Hello all I don't mind helping out with maintaining buildbots / other build machines. Although I don't have a lot of experience with this sort of thing (I usually just ran a Jenkins for my own work), I think it would be a useful way to contribute. Let me know what I should do if you are all fine with this. Thanks Shorya Raj On Wed, Sep 3, 2014 at 2:52 PM, Chris Angelico wrote:

...

Nick Coghlan writes:

...

Sorry, I haven't been a very good maintainer for that buildbot (the

...

...

reason it never graduated to the "stable" list). If you send me your

On Wed, Sep 3, 2014 at 12:47 PM, Stephen J. Turnbull wrote: main public

...

...

SSH key, I can add it (I think - if not, I can ask Luke to do it). Alternatively, CentOS 6 may exhibit the same problem.

I wonder how many of these buildbots could be maintained by the kind of folks who show up on core-mentorship asking "how can I help?"

Just a thought -- I wouldn't be surprised if the reaction is universal horror and the answer is "Are you crazy? Zero! Z-E-R-O!!"

And of course most want to write code, not sysadm.

Maintaining a buildbot isn't hard. (Although one thing I'm not sure of: If my bot goes down for an extended period of time, is any sort of automated email sent to me? I don't often check their status.) But it does mean a measure of trust in some external entity, or else some very careful rules (mainly firewall), which not every coder will know about.

ChrisA _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/rajshorya%40gmail.com

On Thu, Sep 4, 2014 at 3:32 AM, francis wrote:

...

does mean a measure of trust in some external entity, or else some very careful rules (mainly firewall), which not every coder will know about.

Just curious, is there a way to mount the infrastructure the oder way around? One sets a system polling for sources changes, if so it starts a build an sends per e-mail the results to some address.

I'm more talking about how there's a (virtual) machine that I run, executing code sent to me by someone else (the Python devs). That machine needs a certain measure of access to the internet (to fetch code, and to run certain tests), and the easy and naive way to set it up is to give it full access to everything, which is the trust that I spoke of. Firewalling that box so it can't see anything it's not allowed to see is certainly possible, but that requires sysadmin skills, not coder skills, hence my comment. And it's very easy to make a mistake that you won't notice - everything works just fine, because normally that buildbot won't do anything it shouldn't. ChrisA

As mentioned, I don't mind sysadmining a bit, if required. My primary joy would be helping code python, but can't seem to figure out the ideal place to start doing so. Therefore, helping out as sysadmin would be a good start. On Wed, Sep 3, 2014 at 4:58 PM, Cameron Simpson wrote:

On 03Sep2014 11:47, Stephen J. Turnbull wrote:

...

...

Sorry, I haven't been a very good maintainer for that buildbot (the main reason it never graduated to the "stable" list). If you send me your

Nick Coghlan writes: public

...

SSH key, I can add it (I think - if not, I can ask Luke to do it). Alternatively, CentOS 6 may exhibit the same problem.

I wonder how many of these buildbots could be maintained by the kind of folks who show up on core-mentorship asking "how can I help?"

Just a thought -- I wouldn't be surprised if the reaction is universal horror and the answer is "Are you crazy? Zero! Z-E-R-O!!"

And of course most want to write code, not sysadm.

I do both. Happy to help in a small way if wanted.

Cheers, Cameron Simpson

Maintainer's Motto: If we can't fix it, it ain't broke.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ rajshorya%40gmail.com

On 4 September 2014 11:07, Antonio Cavallo wrote:

I wonder if there is any interest in starting to use the opensuse build servers for continuous build and testing on redhat, fedora suse and (I think) debian: that will solve once for all the maintenance issues on those platforms (and provide a reliable build).

It sounds like a good idea in principle, but I suspect there would be a lot of devils in those details. That doesn't make it a bad idea, just something that would need a volunteer to investigate further before we could form a real opinion. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 4 September 2014 15:45, Shorya Raj wrote:

Also, have you considered using Travis? It would allow us to add to the configurations, at least generally...

Many SaaS solution assume the use of specific centralised identity providers, which isn't a function we're prepared to outsource (we know we need to fix our current proliferation of identity silos, but that's a big project and there are other tasks ahead of it on the todo list for the existing infrastructure team). The limiting factor is generally time - various aspects of the current system are clumsy, but they work, so unless someone is particularly keen and willing to work through all the factors that led to the current setup and propose changes (including at least a rough concept of how ongoing maintenance will work), the status quo wins by default. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On 4 September 2014 16:59, Shorya Raj wrote:

...

The limiting factor is generally time - various aspects of the current system are clumsy, but they work, so unless someone is particularly keen and willing to work through all the factors that led to the current setup and propose changes (including at least a rough concept of how ongoing maintenance will work), the status quo wins by default.

If you could outline some of these, I would be willing to have a look at what I could do.

One that seems promising to me would be to experiment with dynamic testing in OpenStack: http://docs.buildbot.net/latest/manual/cfg-buildslaves.html#openstack Creating test VMs on demand avoids a lot of the security issues otherwise incurred by someone setting up a persistent build slave and provides a more consistent test environment. https://wiki.python.org/moin/BuildBot gives details of the test system configuration details. Due to the overlap with the infrastructure team (to make use of the Rackspace VM resources), infrastructure@python.org may be a better list to ask for more details on what OpenStack resources we have available (this idea really sits somewhere between infrastructure and python-dev, as Antoine does most of the BuildBot master maintenance). If we could get something like that running, then we might be able to trim the preconfigured buildbot fleet back to just the non-x86 machines. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

And so what would need to be done to rectify this problem? If there are no objections, I would like to resolve this particular point, at least until better people can be found to do so. On Thu, Sep 4, 2014 at 8:50 PM, A. Cavallo wrote:

Yes there are "details" indeed. But not show stoppers. A prototype can be seen here: http://cclimited.webfactional.com The nice bit is testing will be executed in a total fresh instance, an added bonus

On 4 September 2014 05:16:01 GMT+01:00, Nick Coghlan wrote:

...

On 4 September 2014 11:07, Antonio Cavallo wrote:

...

I wonder if there is any interest in starting to use the opensuse build servers for continuous build and testing on redhat, fedora suse and (I think) debian: that will solve once for all the maintenance issues on those

platforms (and provide a reliable build).

It sounds like a good idea in principle, but I suspect there would be a lot of devils in those details. That doesn't make it a bad idea, just something that would need a volunteer to investigate further

before we could form a real opinion.

Cheers, Nick.

-- Sent from my Android device with K-9 Mail. Please excuse my brevity.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/rajshorya%40gmail.com

Setting up a repo with the code and cleaning a bit here and there. Over the weekend I can put something useable. Nick Coghlan wrote:

On 4 September 2014 18:50, A. Cavallo wrote:

...

Yes there are "details" indeed. But not show stoppers. A prototype can be seen here: http://cclimited.webfactional.com The nice bit is testing will be executed in a total fresh instance, an added bonus

Oh, very nice!

What is needed to get that up and running?

Regards, Nick.

On 8 Sep 2014 04:51, "Antonio Cavallo" wrote:

Ok,

I've put everything under the tools directory of the build server for the

project home:cavallo71:opt-python-interpreters:

https://build.opensuse.org/package/show/home:cavallo71:opt-python-interprete...

To get started you need using the osc client (it uses the same subversion

syntax, so is very simple):

$> osc co home:cavallo71:opt-python-interpreters $> make -f tools/Makefile update (for the 2.7 branch) $> make -f tools/Makefile BRANCH=3 update (for the 3.x branch, I haven't fixed few issue with the build)

I don't think you'd bee able to use as it is because is still linked to my account (few hardcoded paths). But I can help in bootstrapping it if

Thanks for posting this! there's a general consensus on that (maybe having the python project official release build on susebuild?). Integration into the release process will be up to the current release managers (Larry Hastings for 3.4/3.5 and Benjamin Peterson for 2.7). That may be worthwhile as a distro independent way of publishing "opt builds" of Python for Linux platforms (I'd suggest using the softwarecollections.org convention and put them under "/opt/psf/pythonXY"). Note that I think it's worth pursuing this as an integration testing tool, regardless of whether or not we end up incorporating it into the release process.

Then there's the testing bit (it's separate from the python binaries build to insulate the two). It's very similar and I'll post more details when ready.

Thanks again! Regards, Nick.

I hope this helps, Thnaks

Antonio Cavallo wrote:

...

Setting up a repo with the code and cleaning a bit here and there. Over the weekend I can put something useable.

Nick Coghlan wrote:

...

On 4 September 2014 18:50, A. Cavallo wrote:

...

Yes there are "details" indeed. But not show stoppers. A prototype can be seen here: http://cclimited.webfactional.com The nice bit is testing will be executed in a total fresh instance, an added bonus

Oh, very nice!

What is needed to get that up and running?

Regards, Nick.

_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe:

https://mail.python.org/mailman/options/python-dev/a.cavallo%40cavallinux.eu

...

_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com