Continuous Integration
Hello all, NumFOCUS has been working with Continuum Analytics and multiple people in the community on Continuous Integration services for NumPy. Right now the tools we are using are: TeamCity ShiningPandas One great thing about Continuous Integration is that you don't have to make a single decision as long as you have a place to report everything. Continuum Analytics is hosting a TeamCity site and I believe some ShiningPandas sites have been set up. We have some work to do to get the build agents up and running and a single point that communicates the status of all the agents. This email is to just let people know that we are still working on this. Maggie Mari and Bryan van de Ven have been doing the work on the TeamCity side. There are several others who have been involved world wide in setting up build agents and investigating ShiningPandas. Continuum Analytics is still looking for someone who can devote at least 50% of their time to working on these issues as part of their full time job. You can be located anywhere in the world for this. If you are interested send a note to jobs@continuum.io If you have particular reasons why we should choose a particular CI service, please speak up and let your voice be heard. There is still time to make a difference in what we are setting up. Best regards, -Travis
Excerpts from Travis Oliphant's message of mar. mai 01 01:39:26 +0200 2012:
If you have particular reasons why we should choose a particular CI service, please speak up and let your voice be heard. There is still time to make a difference in what we are setting up.
Hi all, What about buildbot? (http://trac.buildbot.net/) I'm using it currently, and like it because is GPL 2, configuration files are powerful Python scripts, and development team is active and dynamic. Best, David
01.05.2012 11:14, David Froger kirjoitti:
Excerpts from Travis Oliphant's message of mar. mai 01 01:39:26 +0200 2012:
If you have particular reasons why we should choose a particular CI service, please speak up and let your voice be heard. There is still time to make a difference in what we are setting up.
Hi all,
What about buildbot? (http://trac.buildbot.net/)
I'm using it currently, and like it because is GPL 2, configuration files are powerful Python scripts, and development team is active and dynamic.
We're currently using it: http://buildbot.scipy.org Although, it hasn't been building automatically for some time now. It has the following shortcomings: - Apparently, for one master instance, only one project. - Last time I looked, Git integration was poor. Maybe this has improved since... Pauli
Hi, On Tue, May 1, 2012 at 9:56 AM, Pauli Virtanen <pav@iki.fi> wrote:
01.05.2012 11:14, David Froger kirjoitti:
Excerpts from Travis Oliphant's message of mar. mai 01 01:39:26 +0200 2012:
If you have particular reasons why we should choose a particular CI service, please speak up and let your voice be heard. There is still time to make a difference in what we are setting up.
Hi all,
What about buildbot? (http://trac.buildbot.net/)
I'm using it currently, and like it because is GPL 2, configuration files are powerful Python scripts, and development team is active and dynamic.
We're currently using it: http://buildbot.scipy.org Although, it hasn't been building automatically for some time now.
It has the following shortcomings:
- Apparently, for one master instance, only one project.
We're building 4 github projects with one buildbot process, and one master.cfg script, but maybe that's not what you meant? http://nipy.bic.berkeley.edu/builders
- Last time I looked, Git integration was poor. Maybe this has improved since...
Our projects use git pollers polling github repos. We did run into random occasional builds stalling at the git update step, but I believe that has been fixed with the latest release. Best, Matthew
Pauli Virtanen <pav <at> iki.fi> writes:
01.05.2012 11:14, David Froger kirjoitti:
Excerpts from Travis Oliphant's message of mar. mai 01 01:39:26 +0200 2012:
If you have particular reasons why we should choose a particular CI
service,
please speak up and let your voice be heard. There is still time to make a difference in what we are setting up.
Hi all,
What about buildbot? (http://trac.buildbot.net/)
I'm using it currently, and like it because is GPL 2, configuration files are powerful Python scripts, and development team is active and dynamic.
We're currently using it: http://buildbot.scipy.org Although, it hasn't been building automatically for some time now.
I've been working on setting up a new buildbot for NumPy. Unfortunately, I don't have much time to work on it, so it's slow going! Right now I am still at the stage of getting NumPy to pass all its tests on the machines I'm using as test slaves. After that, I plan to transfer existing slaves to the new setup, and then maybe ask for new volunteer slave machines (if people think the buildbot setup is useful). One nice feature of the new buildbot setup is that it can build and test a pull request, reporting the results back via a comment on the pull request.
It has the following shortcomings:
- Apparently, for one master instance, only one project.
As others have said, this is no longer true.
- Last time I looked, Git integration was poor. Maybe this has improved since...
Again, as others have said, I think this has been improved since last time you looked. Buildbot can receive github notifications or can poll a repository periodically. On the other hand, I've also been trying ShiningPanda (https:// jenkins.shiningpanda.com/scipy/) because I like the idea of a hosted CI solution. Again, though, I haven't yet got all the tests to pass on ShiningPanda's default Debian 6 setup (and again, that's because of a lack of time). So, I'm working on Buildbot and ShiningPanda from the community side, but am always ready to step aside if someone else has time :) Chris
So, I'm working on Buildbot and ShiningPanda from the community side, but am always ready to step aside if someone else has time :)
Keep it up. Your input and feedback is invaluable. Plus, in this kind of situation, the more the merrier. There are a lot of different agents to test on. -Travis
Chris
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi, On Tue, May 1, 2012 at 3:22 PM, Chris Ball <ceball@gmail.com> wrote:
Pauli Virtanen <pav <at> iki.fi> writes:
01.05.2012 11:14, David Froger kirjoitti:
Excerpts from Travis Oliphant's message of mar. mai 01 01:39:26 +0200 2012:
If you have particular reasons why we should choose a particular CI
service,
please speak up and let your voice be heard. There is still time to make a difference in what we are setting up.
Hi all,
What about buildbot? (http://trac.buildbot.net/)
I'm using it currently, and like it because is GPL 2, configuration files are powerful Python scripts, and development team is active and dynamic.
We're currently using it: http://buildbot.scipy.org Although, it hasn't been building automatically for some time now.
I've been working on setting up a new buildbot for NumPy. Unfortunately, I don't have much time to work on it, so it's slow going! Right now I am still at the stage of getting NumPy to pass all its tests on the machines I'm using as test slaves. After that, I plan to transfer existing slaves to the new setup, and then maybe ask for new volunteer slave machines (if people think the buildbot setup is useful).
One nice feature of the new buildbot setup is that it can build and test a pull request, reporting the results back via a comment on the pull request.
It has the following shortcomings:
- Apparently, for one master instance, only one project.
As others have said, this is no longer true.
- Last time I looked, Git integration was poor. Maybe this has improved since...
Again, as others have said, I think this has been improved since last time you looked. Buildbot can receive github notifications or can poll a repository periodically.
On the other hand, I've also been trying ShiningPanda (https:// jenkins.shiningpanda.com/scipy/) because I like the idea of a hosted CI solution. Again, though, I haven't yet got all the tests to pass on ShiningPanda's default Debian 6 setup (and again, that's because of a lack of time).
The advantage of having some minimal server as master for buildbot is that you can very easily add slaves on which you can trigger builds and get back build logs automatically. As I understand ShiningPanda (never used it, just talked to Fernando), you can push your own build reports up to a ShiningPanda instance, but I hear it's not so easy to trigger builds and upload reports from external slaves. By the way, in case it's interesting, here is our master.cfg for the nipy buildbot; not very pretty, but fairly easy to add a new slave and a new builder: https://github.com/nipy/nibotmi/blob/master/master.cfg and back-of-envelope procedures to set it up: https://github.com/nipy/nibotmi/blob/master/install.rst In particular see 'setting up a buildslave' at the end. Cheers, Matthew
I've been working on setting up a new buildbot for NumPy. Unfortunately, I don't have much time to work on it, so it's slow going! Right now I am still at the stage of getting NumPy to pass all its tests on the machines I'm using as test slaves. After that, I plan to transfer existing slaves to the new setup, and then maybe ask for new volunteer slave machines (if people think the buildbot setup is useful). Hi,
If there are things one can contribute to help the development of the buildbot for NumPy, I would be happy to participate! David
David Froger <david.froger <at> gmail.com> writes:
I've been working on setting up a new buildbot for NumPy. Unfortunately, I don't have much time to work on it, so it's slow going! ... Hi,
If there are things one can contribute to help the development of the buildbot for NumPy, I would be happy to participate!
Great! I've sent you a message off the list so we can coordinate further. Chris
participants (5)
-
Chris Ball -
David Froger -
Matthew Brett -
Pauli Virtanen -
Travis Oliphant