numpy release process, adding a compatibility test step.
Hi, may I suggest an addition to the release process... 'Tests against popular libraries that rely on numpy at the RC stage. Test at least these libraries pass their numpy related tests: matplotlib, scipy, pygame, (insert others here?). The release manage should ask the mailing list for people to test the RC against these libraries to make sure they work ok.' - This will catch problems like the ABI one with the recent release. - It is an explicit request for testing, with concrete tasks that people can do. Rather than a more abstract request for testing. - not much extra work for the release manager. testing work can be distributed to anyone who can try an app, or library with the binaries supplied in the RC stage of a release. - binary installer users can test as well, not just people who build from source. - tests libraries the numpy developers may not be using all the time themselves. On another project, adding this simple step helped prevent a number of bugs which affected other programs and libraries using the project. Before this was added we had a couple of releases which broke a few popular libraries/apps with very simple regressions or bugs. cu,
On Sat, Feb 6, 2010 at 6:44 PM, René Dudfield <renesd@gmail.com> wrote:
Hi,
may I suggest an addition to the release process...
'Tests against popular libraries that rely on numpy at the RC stage. Test at least these libraries pass their numpy related tests: matplotlib, scipy, pygame, (insert others here?). The release manage should ask the mailing list for people to test the RC against these libraries to make sure they work ok.'
- This will catch problems like the ABI one with the recent release. - It is an explicit request for testing, with concrete tasks that people can do. Rather than a more abstract request for testing. - not much extra work for the release manager. testing work can be distributed to anyone who can try an app, or library with the binaries supplied in the RC stage of a release. - binary installer users can test as well, not just people who build from source. - tests libraries the numpy developers may not be using all the time themselves.
On another project, adding this simple step helped prevent a number of bugs which affected other programs and libraries using the project. Before this was added we had a couple of releases which broke a few popular libraries/apps with very simple regressions or bugs.
Thanks for the suggestion, I agree this will be useful. And a lot easier to do than automated testing against other libraries, which will hopefully happen at some point in the future.
Cheers, Ralf
participants (2)
-
Ralf Gommers -
René Dudfield