NumPy 1.7 release delays
Hey all, After some more investigation, I'm not optimistic that we will be able to get a 1.7 release out before SciPy. I would like to get a beta release out by SciPy (or even an rc1 release). But, given the number of code changes and differences between 1.5.x and 1.7, I think we will need an extended beta release stage for 1.7 that will allow as many users as possible to try out the new code base and report back any regressions or backward incompatibilities that need to be fixed before the final release. The fundamental rule I think we have is that "code depending on NumPy that worked with 1.5.x should continue to work with 1.7 without alterations required by the user" This does not mean we can't add new APIs or deprecate old APIs --- but I think that we do have to be much more careful about when deprecated APIs become unavailable. There is a lot of code that assumes the current API. Both code that is in released packages and code that is in "unreleased packages" which we are not even aware of. I don't want to finalize the 1.7 release until we get enough feedback from end-users about the impact of all the changes. This will likely take a longer beta-release period than usual: certainly not until after SciPy where we will make a concerted effort to get people to try the new 1.7 beta and report back on the impact on their code-base. Ondrej is helping out on this effort which I really appreciate. Other people who have time to help with the release effort --- especially in fixing regressions will be greatly appreciated. We are also using this time to 1) setup Continuous Integration services for NumPy using both Jenkins and Travis-CI and 2) migrate the issue tracker to github. Ondrej is heading up #1 and Ray Jones is heading up #2. Please coordinate with them if you'd like to help out on any of those areas. Thanks, -Travis
On Tue, Jun 26, 2012 at 5:43 PM, Travis Oliphant <travis@continuum.io>wrote:
Hey all,
After some more investigation, I'm not optimistic that we will be able to get a 1.7 release out before SciPy. I would like to get a beta release out by SciPy (or even an rc1 release). But, given the number of code changes and differences between 1.5.x and 1.7, I think we will need an extended beta release stage for 1.7 that will allow as many users as possible to try out the new code base and report back any regressions or backward incompatibilities that need to be fixed before the final release.
+1
The fundamental rule I think we have is that "code depending on NumPy that worked with 1.5.x should continue to work with 1.7 without alterations required by the user"
The rule should be 1.6.x imho. Undoing things that were changed in between 1.5.x and 1.6.x makes very little sense; numpy 1.6.0 has been out for over a year.
This does not mean we can't add new APIs or deprecate old APIs --- but I think that we do have to be much more careful about when deprecated APIs become unavailable. There is a lot of code that assumes the current API. Both code that is in released packages and code that is in "unreleased packages" which we are not even aware of.
I think you are mainly talking here about changes that had unintended side-effects, and broke things without anyone realizing that in time. If you read the 1.5.0, 1.6.0 and 1.7.0 release notes, there have been very few actual deprecations. Besides that, we have a long standing policy of removing those things that do get deprecated after one minor release: http://projects.scipy.org/numpy/wiki/ApiDeprecation. If you propose to change that, I suggest discussing it in a separate thread.
I don't want to finalize the 1.7 release until we get enough feedback from end-users about the impact of all the changes. This will likely take a longer beta-release period than usual: certainly not until after SciPy where we will make a concerted effort to get people to try the new 1.7 beta and report back on the impact on their code-base.
Ondrej is helping out on this effort which I really appreciate. Other people who have time to help with the release effort --- especially in fixing regressions will be greatly appreciated.
Did you happen to see https://github.com/numpy/numpy/blob/master/doc/HOWTO_RELEASE.rst.txt? Among other things, it lists a few things that are still to be done (merge doc wiki edits, flip the "raise_warnings" switch) and details on the Wine / MinGW setup that may be useful. I did just spot a mistake there by the way, we're still on MinGW 3.4.5. Cheers, Ralf We are also using this time to 1) setup Continuous Integration services for
NumPy using both Jenkins and Travis-CI and 2) migrate the issue tracker to github. Ondrej is heading up #1 and Ray Jones is heading up #2. Please coordinate with them if you'd like to help out on any of those areas.
Thanks,
-Travis
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Jun 26, 2012, at 2:10 PM, Ralf Gommers wrote:
On Tue, Jun 26, 2012 at 5:43 PM, Travis Oliphant <travis@continuum.io> wrote:
Hey all,
After some more investigation, I'm not optimistic that we will be able to get a 1.7 release out before SciPy. I would like to get a beta release out by SciPy (or even an rc1 release). But, given the number of code changes and differences between 1.5.x and 1.7, I think we will need an extended beta release stage for 1.7 that will allow as many users as possible to try out the new code base and report back any regressions or backward incompatibilities that need to be fixed before the final release.
+1
The fundamental rule I think we have is that "code depending on NumPy that worked with 1.5.x should continue to work with 1.7 without alterations required by the user"
The rule should be 1.6.x imho. Undoing things that were changed in between 1.5.x and 1.6.x makes very little sense; numpy 1.6.0 has been out for over a year.
Unfortunately, I think there are issues we are just now seeing with code that was released in 1.6.x, and there are many people who have not moved forward to 1.6.x yet. The rule should in fact be that code working with NumPy 1.0 should work with 1.7 (except for "bug-fixes"). I realize that with some of the semantics it's going to be hard to be pedantic about the "rule". But, I'm going to be very responsive to users of 1.5.x and even possibly 1.3.x who have code issues in trying to move forward.
This does not mean we can't add new APIs or deprecate old APIs --- but I think that we do have to be much more careful about when deprecated APIs become unavailable. There is a lot of code that assumes the current API. Both code that is in released packages and code that is in "unreleased packages" which we are not even aware of.
I think you are mainly talking here about changes that had unintended side-effects, and broke things without anyone realizing that in time. If you read the 1.5.0, 1.6.0 and 1.7.0 release notes, there have been very few actual deprecations.
Besides that, we have a long standing policy of removing those things that do get deprecated after one minor release: http://projects.scipy.org/numpy/wiki/ApiDeprecation. If you propose to change that, I suggest discussing it in a separate thread.
We need to change that, I think. I feel pretty strongly that we can't just remove APIs after one minor release after observing more of NumPy's use in the wild. APIs should exist for at least 5 years and preferably only change on major releases.
I don't want to finalize the 1.7 release until we get enough feedback from end-users about the impact of all the changes. This will likely take a longer beta-release period than usual: certainly not until after SciPy where we will make a concerted effort to get people to try the new 1.7 beta and report back on the impact on their code-base.
Ondrej is helping out on this effort which I really appreciate. Other people who have time to help with the release effort --- especially in fixing regressions will be greatly appreciated.
Did you happen to see https://github.com/numpy/numpy/blob/master/doc/HOWTO_RELEASE.rst.txt? Among other things, it lists a few things that are still to be done (merge doc wiki edits, flip the "raise_warnings" switch) and details on the Wine / MinGW setup that may be useful. I did just spot a mistake there by the way, we're still on MinGW 3.4.5.
It's nice to have a document like this. Of course, I've seen it. I don't think we will be using Wine and MinGW to do the Windows builds, though. Thanks, -Travis
On Tue, Jun 26, 2012 at 9:20 PM, Travis Oliphant <travis@continuum.io>wrote:
On Jun 26, 2012, at 2:10 PM, Ralf Gommers wrote:
On Tue, Jun 26, 2012 at 5:43 PM, Travis Oliphant <travis@continuum.io>wrote:
Hey all,
After some more investigation, I'm not optimistic that we will be able to get a 1.7 release out before SciPy. I would like to get a beta release out by SciPy (or even an rc1 release). But, given the number of code changes and differences between 1.5.x and 1.7, I think we will need an extended beta release stage for 1.7 that will allow as many users as possible to try out the new code base and report back any regressions or backward incompatibilities that need to be fixed before the final release.
+1
The fundamental rule I think we have is that "code depending on NumPy that worked with 1.5.x should continue to work with 1.7 without alterations required by the user"
The rule should be 1.6.x imho. Undoing things that were changed in between 1.5.x and 1.6.x makes very little sense; numpy 1.6.0 has been out for over a year.
Unfortunately, I think there are issues we are just now seeing with code that was released in 1.6.x, and there are many people who have not moved forward to 1.6.x yet.
Some examples would be nice. A lot of people did move already. And I haven't seen reports of those that tried and got stuck. Also, Debian and Python(x, y) have 1.6.2, EPD has 1.6.1. I think the number of cases we're talking about here is in fact limited. But discussion of those cases is necessary if a change would break 1.6.x. The rule should in fact be that code working with NumPy 1.0 should work
with 1.7 (except for "bug-fixes").
That's a good rule. Hard to ensure for corner cases which didn't have test coverage though. I realize that with some of the semantics it's going to be hard to be
pedantic about the "rule". But, I'm going to be very responsive to users of 1.5.x and even possibly 1.3.x who have code issues in trying to move forward.
This does not mean we can't add new APIs or deprecate old APIs --- but I think that we do have to be much more careful about when deprecated APIs become unavailable. There is a lot of code that assumes the current API. Both code that is in released packages and code that is in "unreleased packages" which we are not even aware of.
I think you are mainly talking here about changes that had unintended side-effects, and broke things without anyone realizing that in time. If you read the 1.5.0, 1.6.0 and 1.7.0 release notes, there have been very few actual deprecations.
Besides that, we have a long standing policy of removing those things that do get deprecated after one minor release: http://projects.scipy.org/numpy/wiki/ApiDeprecation. If you propose to change that, I suggest discussing it in a separate thread.
We need to change that, I think. I feel pretty strongly that we can't just remove APIs after one minor release after observing more of NumPy's use in the wild. APIs should exist for at least 5 years and preferably only change on major releases.
I don't want to finalize the 1.7 release until we get enough feedback from end-users about the impact of all the changes. This will likely take a longer beta-release period than usual: certainly not until after SciPy where we will make a concerted effort to get people to try the new 1.7 beta and report back on the impact on their code-base.
Ondrej is helping out on this effort which I really appreciate. Other people who have time to help with the release effort --- especially in fixing regressions will be greatly appreciated.
Did you happen to see https://github.com/numpy/numpy/blob/master/doc/HOWTO_RELEASE.rst.txt? Among other things, it lists a few things that are still to be done (merge doc wiki edits, flip the "raise_warnings" switch) and details on the Wine / MinGW setup that may be useful. I did just spot a mistake there by the way, we're still on MinGW 3.4.5.
It's nice to have a document like this. Of course, I've seen it. I don't think we will be using Wine and MinGW to do the Windows builds, though.
Any more details? If you are thinking about using MSVC for numpy, will it work with existing scipy and other binaries? Ralf
On 6/26/12 2:48 PM, Ralf Gommers wrote:
Unfortunately, I think there are issues we are just now seeing with code that was released in 1.6.x, and there are many people who have not moved forward to 1.6.x yet.
Some examples would be nice.
I'll bite. Here's an issue that prevents Sage from upgrading to 1.6.2 from 1.5.1: https://github.com/numpy/numpy/issues/291 People are actively working on it (Thanks! Travis commented 13 hours ago about the root of the problem, I think). Thanks, Jason
Unfortunately, I think there are issues we are just now seeing with code that was released in 1.6.x, and there are many people who have not moved forward to 1.6.x yet.
Some examples would be nice. A lot of people did move already. And I haven't seen reports of those that tried and got stuck. Also, Debian and Python(x, y) have 1.6.2, EPD has 1.6.1.
One issues is the one that Sage identified about the array interface regression as noted by Jason. Any other regressions from 1.5.x need to be addressed as well. We'll have to decide on a case-by-case basis if there are issues that conflict with 1.6.x behavior.
It's nice to have a document like this. Of course, I've seen it. I don't think we will be using Wine and MinGW to do the Windows builds, though.
Any more details? If you are thinking about using MSVC for numpy, will it work with existing scipy and other binaries?
It will need to. We need to make sure that whatever we do works for your SciPy binaries. -Travis
Ralf
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
On Tue, Jun 26, 2012 at 10:10 PM, Travis Oliphant <travis@continuum.io>wrote:
Unfortunately, I think there are issues we are just now seeing with code that was released in 1.6.x, and there are many people who have not moved forward to 1.6.x yet.
Some examples would be nice. A lot of people did move already. And I haven't seen reports of those that tried and got stuck. Also, Debian and Python(x, y) have 1.6.2, EPD has 1.6.1.
One issues is the one that Sage identified about the array interface regression as noted by Jason.
That's a good example, and indeed should be fixed. This is clearly a case where no one will be relying on the new behavior; no one wants that object array that's returned in 1.6.x. Any other regressions from 1.5.x need to be addressed as well. We'll
have to decide on a case-by-case basis if there are issues that conflict with 1.6.x behavior.
Sounds good.
It's nice to have a document like this. Of course, I've seen it. I don't think we will be using Wine and MinGW to do the Windows builds, though.
Any more details? If you are thinking about using MSVC for numpy, will it work with existing scipy and other binaries?
It will need to. We need to make sure that whatever we do works for your SciPy binaries.
Great. Please document the new setup, if it's better than the old one it will probably make sense to adopt it for SciPy too. Ralf
On Tue, Jun 26, 2012 at 1:10 PM, Travis Oliphant <travis@continuum.io> wrote:
One issues is the one that Sage identified about the array interface regression as noted by Jason. Any other regressions from 1.5.x need to be addressed as well. We'll have to decide on a case-by-case basis if there are issues that conflict with 1.6.x behavior.
One thing this discussion made me think about, is that it would be great to identify a few key projects that: - use numpy heavily - have reasonably solid test suites and create a special build job that runs *those* test suites periodically. Not necessarily on every last numpy commit, but at least on a reasonable schedule. I think having that kind of information readily available, and with the ability to switch which numpy branch/commit those tests do get run against, could be very valuable as an early warning system for numpy to know if an apparently inconsequential change has unexpected side effects downstream. In IPython we've really benefited greatly from our improved CI infrastructure, but that only goes as far as catching *our own* problems. This kind of downstream integration testing could be very useful. Cheers, f
On Tue, Jun 26, 2012 at 7:59 PM, Fernando Perez <fperez.net@gmail.com> wrote:
On Tue, Jun 26, 2012 at 1:10 PM, Travis Oliphant <travis@continuum.io> wrote:
One issues is the one that Sage identified about the array interface regression as noted by Jason. Any other regressions from 1.5.x need to be addressed as well. We'll have to decide on a case-by-case basis if there are issues that conflict with 1.6.x behavior.
One thing this discussion made me think about, is that it would be great to identify a few key projects that:
- use numpy heavily - have reasonably solid test suites
and create a special build job that runs *those* test suites periodically. Not necessarily on every last numpy commit, but at least on a reasonable schedule.
I think having that kind of information readily available, and with the ability to switch which numpy branch/commit those tests do get run against, could be very valuable as an early warning system for numpy to know if an apparently inconsequential change has unexpected side effects downstream.
In IPython we've really benefited greatly from our improved CI infrastructure, but that only goes as far as catching *our own* problems. This kind of downstream integration testing could be very useful.
+1. Was thinking the same thing. My uninformed opinion from the sidelines: For me, this begged the question of why projects would wait so long and be upgrading 1.5.x -> 1.7.x. it sounded to me like an outreach problem. The whole point of having release candidates is so that downstream users (and especially big public downstream libraries) can test the release candidate and give feedback on any changes that affect them. This feedback step is especially crucial for a project without 100% test coverage (for new code and old)... Putting more restrictions on changes that can be made in releases doesn't seem to me to be the right fix, though, admittedly, numpy is a bit of a different beast than other projects. I would think you would want downstream projects not to wait 2 years to upgrade and skip a couple of minor releases. Skipper
On Tue, Jun 26, 2012 at 8:15 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
On Tue, Jun 26, 2012 at 7:59 PM, Fernando Perez <fperez.net@gmail.com> wrote:
On Tue, Jun 26, 2012 at 1:10 PM, Travis Oliphant <travis@continuum.io> wrote:
One issues is the one that Sage identified about the array interface regression as noted by Jason. Any other regressions from 1.5.x need to be addressed as well. We'll have to decide on a case-by-case basis if there are issues that conflict with 1.6.x behavior.
One thing this discussion made me think about, is that it would be great to identify a few key projects that:
- use numpy heavily - have reasonably solid test suites
and create a special build job that runs *those* test suites periodically. Not necessarily on every last numpy commit, but at least on a reasonable schedule.
I think having that kind of information readily available, and with the ability to switch which numpy branch/commit those tests do get run against, could be very valuable as an early warning system for numpy to know if an apparently inconsequential change has unexpected side effects downstream.
In IPython we've really benefited greatly from our improved CI infrastructure, but that only goes as far as catching *our own* problems. This kind of downstream integration testing could be very useful.
+1. Was thinking the same thing.
My uninformed opinion from the sidelines: For me, this begged the question of why projects would wait so long and be upgrading 1.5.x -> 1.7.x. it sounded to me like an outreach problem. The whole point of having release candidates is so that downstream users (and especially big public downstream libraries) can test the release candidate and give feedback on any changes that affect them. This feedback step is especially crucial for a project without 100% test coverage (for new code and old)... Putting more restrictions on changes that can be made in releases doesn't seem to me to be the right fix, though, admittedly, numpy is a bit of a different beast than other projects. I would think you would want downstream projects not to wait 2 years to upgrade and skip a couple of minor releases.
Skipper _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
+1. We've begun running pandas's test suite internally against NumPy git master on Jenkins. It has already turned up bugs and behavior changes in a few short weeks. We should definitely do this on a more grand scale (especially since pandas 0.8.0 is now littered with hacks around NumPy 1.6 datetime bugs. fortunately nothing was fatally broken but it came close). - Wes
On 06/26/2012 06:15 PM, Skipper Seabold wrote:
My uninformed opinion from the sidelines: For me, this begged the question of why projects would wait so long and be upgrading 1.5.x -> 1.7.x. it sounded to me like an outreach problem. lenny: none squeeze: 1.4.1 wheezy: 1.6.2
hardy: 1.0.4 lucid: 1.3.0 precise: 1.6.1 Even those of us who are developing downstream aren't necessarily rushing to hand-roll all the dependencies. Our users certainly aren't and the alternatives of abandoning support for earlier versions of numpy or having a bunch of version tests aren't appealing, either. I am somewhat surprised by the notion of breaking API compatibility without changing the major version number. -- Jonathan Niehof ISR-3 Space Data Systems Los Alamos National Laboratory MS-D466 Los Alamos, NM 87545 Phone: 505-667-9595 email: jniehof@lanl.gov Correspondence / Technical data or Software Publicly Available
On Tue, Jun 26, 2012 at 4:59 PM, Fernando Perez <fperez.net@gmail.com> wrote:
On Tue, Jun 26, 2012 at 1:10 PM, Travis Oliphant <travis@continuum.io> wrote:
One issues is the one that Sage identified about the array interface regression as noted by Jason. Any other regressions from 1.5.x need to be addressed as well. We'll have to decide on a case-by-case basis if there are issues that conflict with 1.6.x behavior.
One thing this discussion made me think about, is that it would be great to identify a few key projects that:
- use numpy heavily - have reasonably solid test suites
and create a special build job that runs *those* test suites periodically. Not necessarily on every last numpy commit, but at least on a reasonable schedule.
I think having that kind of information readily available, and with the ability to switch which numpy branch/commit those tests do get run against, could be very valuable as an early warning system for numpy to know if an apparently inconsequential change has unexpected side effects downstream.
I think that is a great idea. It would simply recompile numpy, but leave the other library intact, and then run some tests on the other library, which could be as simple as "import h5py", or more complicated.
In IPython we've really benefited greatly from our improved CI infrastructure, but that only goes as far as catching *our own*
Do you use anything else besides Travis CI? I donated money to them and they enabled pull request testing for SymPy and it's invaluable. We also use our custom sympy-bot (https://github.com/sympy/sympy-bot) to test pull request, but now when Travis can do that, we might just use that. NumPy now has Travis for both master and pull requests and so it is pretty well covered. I need to setup some Jenkins instances for Windows (and Mac) testing, and then we can also add linux Jenkins instance to test numpy against a few other libraries.
problems. This kind of downstream integration testing could be very useful.
Ondrej
On Tue, Jun 26, 2012 at 5:40 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
Do you use anything else besides Travis CI?
Yes, we use both Shining Panda and Travis CI: https://jenkins.shiningpanda.com/ipython/ http://travis-ci.org/#!/ipython/ipython The SP setup is more complete, including Mac and Windows bots.
I donated money to them and they enabled pull request testing for SymPy and it's invaluable. We also use our custom sympy-bot (https://github.com/sympy/sympy-bot) to test pull request, but now when Travis can do that, we might just use that.
We have a version of that: after Aaron Meurer gave us an invaluable and detailed report on how you guys used it, Thomas Kluyver built for us our new test_pr script: https://github.com/ipython/ipython/blob/master/tools/test_pr.py which we regularly use now in most PRs, e.g.: https://github.com/ipython/ipython/pull/2015#issuecomment-6566387 It has proven to be *extremely* useful. This is some of the infrastructure that I hope we'll gradually start using across all the projects (the topic of some of the threads in the numfocus list). In IPython, our ability to rapidly absorb code has improved tremendously in part thanks to the smooth workflow these tools give us; just in the month of June we've merged 116 PRs totaling over 400 commits: (master)dreamweaver[ipython]> git log --oneline --since 2012-06-01 | grep "Merge pull request" | wc -l 116 (master)dreamweaver[ipython]> git log --oneline --since 2012-06-01 | wc -l 438 There's no way to keep that pace unless we can really trust our testing machinery to let us know what's safe by the time we get to code review. As our tools mature, I really hope we'll start using them more across different projects, because the benefit they provide is undeniable. Cheers, f
Hi, On Tue, Jun 26, 2012 at 6:00 PM, Fernando Perez <fperez.net@gmail.com> wrote:
On Tue, Jun 26, 2012 at 5:40 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
Do you use anything else besides Travis CI?
Yes, we use both Shining Panda and Travis CI:
https://jenkins.shiningpanda.com/ipython/ http://travis-ci.org/#!/ipython/ipython
The SP setup is more complete, including Mac and Windows bots.
I donated money to them and they enabled pull request testing for SymPy and it's invaluable. We also use our custom sympy-bot (https://github.com/sympy/sympy-bot) to test pull request, but now when Travis can do that, we might just use that.
We have a version of that: after Aaron Meurer gave us an invaluable and detailed report on how you guys used it, Thomas Kluyver built for us our new test_pr script:
https://github.com/ipython/ipython/blob/master/tools/test_pr.py
which we regularly use now in most PRs, e.g.:
https://github.com/ipython/ipython/pull/2015#issuecomment-6566387
It has proven to be *extremely* useful.
This is some of the infrastructure that I hope we'll gradually start using across all the projects (the topic of some of the threads in the numfocus list). In IPython, our ability to rapidly absorb code has improved tremendously in part thanks to the smooth workflow these tools give us; just in the month of June we've merged 116 PRs totaling over 400 commits:
(master)dreamweaver[ipython]> git log --oneline --since 2012-06-01 | grep "Merge pull request" | wc -l 116
(master)dreamweaver[ipython]> git log --oneline --since 2012-06-01 | wc -l 438
There's no way to keep that pace unless we can really trust our testing machinery to let us know what's safe by the time we get to code review.
As our tools mature, I really hope we'll start using them more across different projects, because the benefit they provide is undeniable.
We (nipy'ers) are heavy users of numpy and scipy. We use travis-ci for testing individual commits to personal repos: https://github.com/nipy/nibabel/blob/master/.travis.yml (using standard travis-ci python test machinery, multiple python versions) https://github.com/nipy/dipy/blob/master/.travis.yml https://github.com/nipy/nipy/blob/master/.travis.yml (using a hack to test against a system python, to avoid multiple compiles of numpy / scipy). We've also been discussing numpy / scipy compiles on the Travis-CI mailing list : https://groups.google.com/d/topic/travis-ci/uJgu35XKdmI/discussion. For the main repos we use buildbot and test on: Ubuntu Maverick 32-bit Debian sid 64-bit OSX 10.4 PPC OSX 10.5 Intel Debian wheezy PPC Debian squeeze ARM (a Raspberry PI no less) WIndows XP 32 bit SPARC (courtesy of our friends at NeuroDebian) http://nipy.bic.berkeley.edu/builders We've found several issues with numpy using these, and I've fed them back as I found them, http://projects.scipy.org/numpy/ticket/2076 http://projects.scipy.org/numpy/ticket/2077 http://projects.scipy.org/numpy/ticket/2174 They are particularly useful for difficult to reproduce problems because they test often and leave a record that we can point to. As I've said before, y'all are welcome to use these machines for numpy builds / tests. Best, Matthew
For the main repos we use buildbot and test on:
Ubuntu Maverick 32-bit Debian sid 64-bit OSX 10.4 PPC OSX 10.5 Intel Debian wheezy PPC Debian squeeze ARM (a Raspberry PI no less) WIndows XP 32 bit SPARC (courtesy of our friends at NeuroDebian)
http://nipy.bic.berkeley.edu/builders
We've found several issues with numpy using these, and I've fed them back as I found them,
http://projects.scipy.org/numpy/ticket/2076 http://projects.scipy.org/numpy/ticket/2077 http://projects.scipy.org/numpy/ticket/2174
They are particularly useful for difficult to reproduce problems because they test often and leave a record that we can point to. As I've said before, y'all are welcome to use these machines for numpy builds / tests.
Now that Ondrej is working on getting continuous integration up for NumPy, I would encourage him to take you up on that offer. Can these machines run a Jenkins slave? Having periodic tests of Sage, Pandas, matplotlib, scipy, and other projects is a major priority and really critical before we can really talk about how to migrate the APIs. Thankfully, Ondrej is available to help get this project started and working this summer. -Travis
Best,
Matthew _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi, On Tue, Jun 26, 2012 at 8:13 PM, Travis Oliphant <travis@continuum.io> wrote:
For the main repos we use buildbot and test on:
Ubuntu Maverick 32-bit Debian sid 64-bit OSX 10.4 PPC OSX 10.5 Intel Debian wheezy PPC Debian squeeze ARM (a Raspberry PI no less) WIndows XP 32 bit SPARC (courtesy of our friends at NeuroDebian)
http://nipy.bic.berkeley.edu/builders
We've found several issues with numpy using these, and I've fed them back as I found them,
http://projects.scipy.org/numpy/ticket/2076 http://projects.scipy.org/numpy/ticket/2077 http://projects.scipy.org/numpy/ticket/2174
They are particularly useful for difficult to reproduce problems because they test often and leave a record that we can point to. As I've said before, y'all are welcome to use these machines for numpy builds / tests.
Now that Ondrej is working on getting continuous integration up for NumPy, I would encourage him to take you up on that offer. Can these machines run a Jenkins slave?
I believe so, but haven't tested. Best, Matthew
On 6/26/2012 8:13 PM, Travis Oliphant wrote:
For the main repos we use buildbot and test on:
Ubuntu Maverick 32-bit Debian sid 64-bit OSX 10.4 PPC OSX 10.5 Intel Debian wheezy PPC Debian squeeze ARM (a Raspberry PI no less) WIndows XP 32 bit SPARC (courtesy of our friends at NeuroDebian)
http://nipy.bic.berkeley.edu/builders
We've found several issues with numpy using these, and I've fed them back as I found them,
http://projects.scipy.org/numpy/ticket/2076 http://projects.scipy.org/numpy/ticket/2077 http://projects.scipy.org/numpy/ticket/2174
They are particularly useful for difficult to reproduce problems because they test often and leave a record that we can point to. As I've said before, y'all are welcome to use these machines for numpy builds / tests.
Now that Ondrej is working on getting continuous integration up for NumPy, I would encourage him to take you up on that offer. Can these machines run a Jenkins slave?
Having periodic tests of Sage, Pandas, matplotlib, scipy, and other projects is a major priority and really critical before we can really talk about how to migrate the APIs. Thankfully, Ondrej is available to help get this project started and working this summer.
-Travis
FWIW: I can relatively easy (batch script) build numpy from github and run the test suites of many packages available at <http://www.lfd.uci.edu/~gohlke/pythonlibs/> against it. For example at <http://www.lfd.uci.edu/~gohlke/pythonlibs/tests/numpy-MKL-1.7.0.dev-66bd39f-...> are the test results of assimulo, bitarray, bottleneck, h5py, matplotlib, numexpr, pandas, pygame, scipy, skimage, sklearn, statsmodels, and pytables, built against numpy-1.6.x and run against numpy-1.7.0.dev-66bd39f on win-amd64-py2.7. Christoph
Hi, On Wed, Jun 27, 2012 at 12:38 AM, Christoph Gohlke <cgohlke@uci.edu> wrote:
On 6/26/2012 8:13 PM, Travis Oliphant wrote:
For the main repos we use buildbot and test on:
Ubuntu Maverick 32-bit Debian sid 64-bit OSX 10.4 PPC OSX 10.5 Intel Debian wheezy PPC Debian squeeze ARM (a Raspberry PI no less) WIndows XP 32 bit SPARC (courtesy of our friends at NeuroDebian)
http://nipy.bic.berkeley.edu/builders
We've found several issues with numpy using these, and I've fed them back as I found them,
http://projects.scipy.org/numpy/ticket/2076 http://projects.scipy.org/numpy/ticket/2077 http://projects.scipy.org/numpy/ticket/2174
They are particularly useful for difficult to reproduce problems because they test often and leave a record that we can point to. As I've said before, y'all are welcome to use these machines for numpy builds / tests.
Now that Ondrej is working on getting continuous integration up for NumPy, I would encourage him to take you up on that offer. Can these machines run a Jenkins slave?
Having periodic tests of Sage, Pandas, matplotlib, scipy, and other projects is a major priority and really critical before we can really talk about how to migrate the APIs. Thankfully, Ondrej is available to help get this project started and working this summer.
-Travis
FWIW: I can relatively easy (batch script) build numpy from github and run the test suites of many packages available at <http://www.lfd.uci.edu/~gohlke/pythonlibs/> against it.
For example at <http://www.lfd.uci.edu/~gohlke/pythonlibs/tests/numpy-MKL-1.7.0.dev-66bd39f-...> are the test results of assimulo, bitarray, bottleneck, h5py, matplotlib, numexpr, pandas, pygame, scipy, skimage, sklearn, statsmodels, and pytables, built against numpy-1.6.x and run against numpy-1.7.0.dev-66bd39f on win-amd64-py2.7.
Thanks - that's very helpful. Do you have your build system documented somewhere? Is it easy to replicate do you think? Cheers, Matthew
On Sat, Jun 30, 2012 at 1:52 AM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Wed, Jun 27, 2012 at 12:38 AM, Christoph Gohlke <cgohlke@uci.edu> wrote:
On 6/26/2012 8:13 PM, Travis Oliphant wrote:
For the main repos we use buildbot and test on:
Ubuntu Maverick 32-bit Debian sid 64-bit OSX 10.4 PPC OSX 10.5 Intel Debian wheezy PPC Debian squeeze ARM (a Raspberry PI no less) WIndows XP 32 bit SPARC (courtesy of our friends at NeuroDebian)
http://nipy.bic.berkeley.edu/builders
We've found several issues with numpy using these, and I've fed them back as I found them,
http://projects.scipy.org/numpy/ticket/2076 http://projects.scipy.org/numpy/ticket/2077 http://projects.scipy.org/numpy/ticket/2174
They are particularly useful for difficult to reproduce problems because they test often and leave a record that we can point to. As I've said before, y'all are welcome to use these machines for numpy builds / tests.
Now that Ondrej is working on getting continuous integration up for NumPy, I would encourage him to take you up on that offer. Can these machines run a Jenkins slave?
Having periodic tests of Sage, Pandas, matplotlib, scipy, and other projects is a major priority and really critical before we can really talk about how to migrate the APIs. Thankfully, Ondrej is available to help get this project started and working this summer.
-Travis
FWIW: I can relatively easy (batch script) build numpy from github and run the test suites of many packages available at <http://www.lfd.uci.edu/~gohlke/pythonlibs/> against it.
For example at <http://www.lfd.uci.edu/~gohlke/pythonlibs/tests/numpy-MKL-1.7.0.dev-66bd39f-...> are the test results of assimulo, bitarray, bottleneck, h5py, matplotlib, numexpr, pandas, pygame, scipy, skimage, sklearn, statsmodels, and pytables, built against numpy-1.6.x and run against numpy-1.7.0.dev-66bd39f on win-amd64-py2.7.
Thanks - that's very helpful.
same here. I just saw a python 3.2.3 bug in statsmodels. Josef
Do you have your build system documented somewhere? Is it easy to replicate do you think?
Cheers,
Matthew _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
Hi Matthew, On Tue, Jun 26, 2012 at 7:07 PM, Matthew Brett <matthew.brett@gmail.com> wrote:
Hi,
On Tue, Jun 26, 2012 at 6:00 PM, Fernando Perez <fperez.net@gmail.com> wrote:
On Tue, Jun 26, 2012 at 5:40 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
Do you use anything else besides Travis CI?
Yes, we use both Shining Panda and Travis CI:
https://jenkins.shiningpanda.com/ipython/ http://travis-ci.org/#!/ipython/ipython
The SP setup is more complete, including Mac and Windows bots.
I donated money to them and they enabled pull request testing for SymPy and it's invaluable. We also use our custom sympy-bot (https://github.com/sympy/sympy-bot) to test pull request, but now when Travis can do that, we might just use that.
We have a version of that: after Aaron Meurer gave us an invaluable and detailed report on how you guys used it, Thomas Kluyver built for us our new test_pr script:
https://github.com/ipython/ipython/blob/master/tools/test_pr.py
which we regularly use now in most PRs, e.g.:
https://github.com/ipython/ipython/pull/2015#issuecomment-6566387
It has proven to be *extremely* useful.
This is some of the infrastructure that I hope we'll gradually start using across all the projects (the topic of some of the threads in the numfocus list). In IPython, our ability to rapidly absorb code has improved tremendously in part thanks to the smooth workflow these tools give us; just in the month of June we've merged 116 PRs totaling over 400 commits:
(master)dreamweaver[ipython]> git log --oneline --since 2012-06-01 | grep "Merge pull request" | wc -l 116
(master)dreamweaver[ipython]> git log --oneline --since 2012-06-01 | wc -l 438
There's no way to keep that pace unless we can really trust our testing machinery to let us know what's safe by the time we get to code review.
As our tools mature, I really hope we'll start using them more across different projects, because the benefit they provide is undeniable.
We (nipy'ers) are heavy users of numpy and scipy.
We use travis-ci for testing individual commits to personal repos:
https://github.com/nipy/nibabel/blob/master/.travis.yml
(using standard travis-ci python test machinery, multiple python versions)
https://github.com/nipy/dipy/blob/master/.travis.yml https://github.com/nipy/nipy/blob/master/.travis.yml
(using a hack to test against a system python, to avoid multiple compiles of numpy / scipy). We've also been discussing numpy / scipy compiles on the Travis-CI mailing list : https://groups.google.com/d/topic/travis-ci/uJgu35XKdmI/discussion.
For the main repos we use buildbot and test on:
Ubuntu Maverick 32-bit Debian sid 64-bit OSX 10.4 PPC OSX 10.5 Intel Debian wheezy PPC Debian squeeze ARM (a Raspberry PI no less) WIndows XP 32 bit SPARC (courtesy of our friends at NeuroDebian)
http://nipy.bic.berkeley.edu/builders
We've found several issues with numpy using these, and I've fed them back as I found them,
http://projects.scipy.org/numpy/ticket/2076 http://projects.scipy.org/numpy/ticket/2077 http://projects.scipy.org/numpy/ticket/2174
They are particularly useful for difficult to reproduce problems because they test often and leave a record that we can point to. As I've said before, y'all are welcome to use these machines for numpy builds / tests.
This is amazing, thanks a lot for the email. I'll talk to you offlist. Thanks, Ondrej
Hi Fernando, On Tue, Jun 26, 2012 at 6:00 PM, Fernando Perez <fperez.net@gmail.com> wrote:
On Tue, Jun 26, 2012 at 5:40 PM, Ondřej Čertík <ondrej.certik@gmail.com> wrote:
Do you use anything else besides Travis CI?
Yes, we use both Shining Panda and Travis CI:
With NumPy, I am still there in the waitinglist, number 92. So I guess it will take a while.
http://travis-ci.org/#!/ipython/ipython
The SP setup is more complete, including Mac and Windows bots.
I donated money to them and they enabled pull request testing for SymPy and it's invaluable. We also use our custom sympy-bot (https://github.com/sympy/sympy-bot) to test pull request, but now when Travis can do that, we might just use that.
We have a version of that: after Aaron Meurer gave us an invaluable and detailed report on how you guys used it, Thomas Kluyver built for us our new test_pr script:
https://github.com/ipython/ipython/blob/master/tools/test_pr.py
which we regularly use now in most PRs, e.g.:
https://github.com/ipython/ipython/pull/2015#issuecomment-6566387
It has proven to be *extremely* useful.
I see, yes.
This is some of the infrastructure that I hope we'll gradually start using across all the projects (the topic of some of the threads in the numfocus list). In IPython, our ability to rapidly absorb code has improved tremendously in part thanks to the smooth workflow these tools give us; just in the month of June we've merged 116 PRs totaling over 400 commits:
(master)dreamweaver[ipython]> git log --oneline --since 2012-06-01 | grep "Merge pull request" | wc -l 116
(master)dreamweaver[ipython]> git log --oneline --since 2012-06-01 | wc -l 438
There's no way to keep that pace unless we can really trust our testing machinery to let us know what's safe by the time we get to code review.
As our tools mature, I really hope we'll start using them more across different projects, because the benefit they provide is undeniable.
Thanks for the write up. Yes, I have exactly the same experience with SymPy's pull requests. So I have personally spent a lot of time trying to streamline the process and making sure that we can trust it and so on. My bet is that Travis CI will be *the* tool of choice for most projects at least on linux, to test the pull requests. Ondrej
Some examples would be nice. A lot of people did move already. And I haven't seen reports of those that tried and got stuck. Also, Debian and Python(x, y) have 1.6.2, EPD has 1.6.1.
In my company, the numpy for our production python install is well behind 1.6. In the world of trading, the upgrade cycle can be slow, because when people have production trading systems that are working and running stably, they have little or no incentive to upgrade. I know Travis has been doing a lot of consulting inside major banks and investment houses, and these are probably the kinds of people he sees regularly. You also have a fair amount of personnel turnover over the years, so that the developer who wrote the trading system may have moved on, and an upgrade which breaks the code is difficult to repair because the original developers are gone. So people are loathe to upgrade. It is certainly true that deprecations that have lived for a single point release cycle have not been vetted by a large part of the user community. In my group, we try to stay as close to the bleeding edge as possible so as to not fall behind and make an upgrade painful, but we are not the rule. JDH
On Wed, Jun 27, 2012 at 7:20 AM, John Hunter <jdh2358@gmail.com> wrote:
Some examples would be nice. A lot of people did move already. And I haven't seen reports of those that tried and got stuck. Also, Debian and Python(x, y) have 1.6.2, EPD has 1.6.1.
In my company, the numpy for our production python install is well behind 1.6. In the world of trading, the upgrade cycle can be slow, because when people have production trading systems that are working and running stably, they have little or no incentive to upgrade. I know Travis has been doing a lot of consulting inside major banks and investment houses, and these are probably the kinds of people he sees regularly. You also have a fair amount of personnel turnover over the years, so that the developer who wrote the trading system may have moved on, and an upgrade which breaks the code is difficult to repair because the original developers are gone. So people are loathe to upgrade. It is certainly true that deprecations that have lived for a single point release cycle have not been vetted by a large part of the user community.
I'd also venture a guess that many of those installations don't have adequate test suites.
In my group, we try to stay as close to the bleeding edge as possible so as to not fall behind and make an upgrade painful, but we are not the rule.
Chuck
On Wed, Jun 27, 2012 at 9:33 AM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 27, 2012 at 7:20 AM, John Hunter <jdh2358@gmail.com> wrote:
because the original developers are gone. So people are loathe to upgrade. It is certainly true that deprecations that have lived for a single point release cycle have not been vetted by a large part of the user community.
I'd also venture a guess that many of those installations don't have adequate test suites.
Depends how you define "adequate". If these companies stopped being able to make money, model science, or serve up web applications due to the lack of a test suite, then they would immediately put all their efforts on it. But for users for whom Numpy (and software in general) is merely a means to an end, the management of technical debt and future technology risk is merely one component of all the risk factors facing the organization. Every hour spent managing code and technical debt is an hour of lost business opportunity, and that balance is very conscientiously weighed and oftentimes the decision is not in the direction of quality of software process. In my experience, it's a toss up.. most people have reasonable unit tests and small integration tests, but large scale smoke tests can be very difficult to maintain or to justify to upper management. Because Numpy can be both a component of larger software or a direct tool in its own right, I've found that it makes a big difference whether an organization sees code as a means to an end, or an ends unto itself. -Peter
On Wed, Jun 27, 2012 at 8:43 AM, Peter Wang <pwang@continuum.io> wrote:
On Wed, Jun 27, 2012 at 9:33 AM, Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Jun 27, 2012 at 7:20 AM, John Hunter <jdh2358@gmail.com> wrote:
because the original developers are gone. So people are loathe to upgrade. It is certainly true that deprecations that have lived for a single point release cycle have not been vetted by a large part of the user community.
I'd also venture a guess that many of those installations don't have adequate test suites.
Depends how you define "adequate". If these companies stopped being able to make money, model science, or serve up web applications due to the lack of a test suite, then they would immediately put all their efforts on it. But for users for whom Numpy (and software in general) is merely a means to an end, the management of technical debt and future technology risk is merely one component of all the risk factors facing the organization. Every hour spent managing code and technical debt is an hour of lost business opportunity, and that balance is very conscientiously weighed and oftentimes the decision is not in the direction of quality of software process.
In my experience, it's a toss up.. most people have reasonable unit tests and small integration tests, but large scale smoke tests can be very difficult to maintain or to justify to upper management. Because Numpy can be both a component of larger software or a direct tool in its own right, I've found that it makes a big difference whether an organization sees code as a means to an end, or an ends unto itself.
Yep. What I meant by adequate was adequate for safely upgrading/refactoring. My experience is that folks will stay with what they have as long as it gets the job done. Competition can drive changes, but even that can be an unreliable prod as it may take several years before losing an edge begins to hurt. Chuck
participants (14)
-
Charles R Harris -
Christoph Gohlke -
Fernando Perez -
Jason Grout -
John Hunter -
Jonathan T. Niehof -
josef.pktd＠gmail.com -
Matthew Brett -
Ondřej Čertík -
Peter Wang -
Ralf Gommers -
Skipper Seabold -
Travis Oliphant -
Wes McKinney