ANN: SciPy 0.9.0 release candidate 2
Hi, I am pleased to announce the availability of the second release candidate of SciPy 0.9.0. This will be the first SciPy release to include support for Python 3 (all modules except scipy.weave), as well as for Python 2.7. Due to the Sourceforge outage I am not able to put binaries on the normal download site right now, that will probably only happen in a week or so. If you want to try the RC now please build from svn, and report any issues. Changes since release candidate 1: - fixes for build problems with MSVC + MKL (#1210, #1376) - fix pilutil test to work with numpy master branch - fix constants.codata to be backwards-compatible Enjoy, Ralf
Hi, On Mon, 31 Jan 2011 09:50:33 +0800, Ralf Gommers wrote:
I am pleased to announce the availability of the second release candidate of SciPy 0.9.0. This will be the first SciPy release to include support for Python 3 (all modules except scipy.weave), as well as for Python 2.7.
Due to the Sourceforge outage I am not able to put binaries on the normal download site right now, that will probably only happen in a week or so. If you want to try the RC now please build from svn, and report any issues.
When you build from SVN, please test the latest 0.9.x branch (and not the 0.9.0rc2 tag): http://svn.scipy.org/svn/scipy/branches/0.9.x It contains an additional last-minute bug fix, which we will want to have in 0.9.0. -- Pauli Virtanen
On Sun, Jan 30, 2011 at 6:50 PM, Ralf Gommers <ralf.gommers@googlemail.com>wrote:
Hi,
I am pleased to announce the availability of the second release candidate of SciPy 0.9.0. This will be the first SciPy release to include support for Python 3 (all modules except scipy.weave), as well as for Python 2.7.
Due to the Sourceforge outage I am not able to put binaries on the normal download site right now, that will probably only happen in a week or so. If you want to try the RC now please build from svn, and report any issues.
Changes since release candidate 1: - fixes for build problems with MSVC + MKL (#1210, #1376) - fix pilutil test to work with numpy master branch - fix constants.codata to be backwards-compatible
I think there should be a fix for ndarray also, the problem with type 5 (int) not being recognized is that it checks for Int32, which on some (all?) 32 bit platforms is a long (7) rather than an int. I think this is a bug and it will cause problems with numpy 1.6. Chuck
On Mon, Jan 31, 2011 at 11:01 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Sun, Jan 30, 2011 at 6:50 PM, Ralf Gommers < ralf.gommers@googlemail.com> wrote:
Hi,
I am pleased to announce the availability of the second release candidate of SciPy 0.9.0. This will be the first SciPy release to include support for Python 3 (all modules except scipy.weave), as well as for Python 2.7.
Due to the Sourceforge outage I am not able to put binaries on the normal download site right now, that will probably only happen in a week or so. If you want to try the RC now please build from svn, and report any issues.
Changes since release candidate 1: - fixes for build problems with MSVC + MKL (#1210, #1376) - fix pilutil test to work with numpy master branch - fix constants.codata to be backwards-compatible
I think there should be a fix for ndarray also, the problem with type 5 (int) not being recognized is that it checks for Int32, which on some (all?) 32 bit platforms is a long (7) rather than an int. I think this is a bug and it will cause problems with numpy 1.6.
You mean ndimage I guess. I don't really understand your explanation, if that's the case then those ndimage tests should have been failing before, right? If this needs to be fixed in scipy then we need an RC3. I would like to get that out by the 12th or so if possible. Cheers, Ralf
On Tue, Feb 1, 2011 at 9:04 PM, Ralf Gommers <ralf.gommers@googlemail.com>wrote:
On Mon, Jan 31, 2011 at 11:01 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Sun, Jan 30, 2011 at 6:50 PM, Ralf Gommers < ralf.gommers@googlemail.com> wrote:
Hi,
I am pleased to announce the availability of the second release candidate of SciPy 0.9.0. This will be the first SciPy release to include support for Python 3 (all modules except scipy.weave), as well as for Python 2.7.
Due to the Sourceforge outage I am not able to put binaries on the normal download site right now, that will probably only happen in a week or so. If you want to try the RC now please build from svn, and report any issues.
Changes since release candidate 1: - fixes for build problems with MSVC + MKL (#1210, #1376) - fix pilutil test to work with numpy master branch - fix constants.codata to be backwards-compatible
I think there should be a fix for ndarray also, the problem with type 5 (int) not being recognized is that it checks for Int32, which on some (all?) 32 bit platforms is a long (7) rather than an int. I think this is a bug and it will cause problems with numpy 1.6.
You mean ndimage I guess. I don't really understand your explanation, if that's the case then those ndimage tests should have been failing before, right?
If this needs to be fixed in scipy then we need an RC3. I would like to get that out by the 12th or so if possible.
I think it needs to be fixed as it is a bug. The problem is that ndimage checks types, but only by Int16, Int32, etc. On 32 bit systems and windows Int32 is a long (type# 7) and integers (type# 5), are treated as unrecognized types even though they are the same thing. Ndimage should accept both types. Chuck
On Wed, Feb 2, 2011 at 12:57 PM, Charles R Harris <charlesr.harris@gmail.com
wrote:
On Tue, Feb 1, 2011 at 9:04 PM, Ralf Gommers <ralf.gommers@googlemail.com>wrote:
On Mon, Jan 31, 2011 at 11:01 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Sun, Jan 30, 2011 at 6:50 PM, Ralf Gommers < ralf.gommers@googlemail.com> wrote:
Hi,
I am pleased to announce the availability of the second release candidate of SciPy 0.9.0. This will be the first SciPy release to include support for Python 3 (all modules except scipy.weave), as well as for Python 2.7.
Due to the Sourceforge outage I am not able to put binaries on the normal download site right now, that will probably only happen in a week or so. If you want to try the RC now please build from svn, and report any issues.
Changes since release candidate 1: - fixes for build problems with MSVC + MKL (#1210, #1376) - fix pilutil test to work with numpy master branch - fix constants.codata to be backwards-compatible
I think there should be a fix for ndarray also, the problem with type 5 (int) not being recognized is that it checks for Int32, which on some (all?) 32 bit platforms is a long (7) rather than an int. I think this is a bug and it will cause problems with numpy 1.6.
You mean ndimage I guess. I don't really understand your explanation, if that's the case then those ndimage tests should have been failing before, right?
If this needs to be fixed in scipy then we need an RC3. I would like to get that out by the 12th or so if possible.
I think it needs to be fixed as it is a bug. The problem is that ndimage checks types, but only by Int16, Int32, etc. On 32 bit systems and windows Int32 is a long (type# 7) and integers (type# 5), are treated as unrecognized types even though they are the same thing. Ndimage should accept both types.
I'm still a bit puzzled about why it's only failing after the recent
changes in numpy master. Sounds like it's a bug, but it's not a regression and we are at rc2 right now. Do you (or does someone else) have time to fix it in the next week or so? The other option is to fix it in 0.9.1, which can come out at the same time as numpy 1.6.0. Cheers, Ralf
On Wed, Feb 2, 2011 at 1:11 PM, Ralf Gommers <ralf.gommers@googlemail.com>wrote:
On Wed, Feb 2, 2011 at 12:57 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Tue, Feb 1, 2011 at 9:04 PM, Ralf Gommers <ralf.gommers@googlemail.com
wrote:
On Mon, Jan 31, 2011 at 11:01 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
I think there should be a fix for ndarray also, the problem with type 5 (int) not being recognized is that it checks for Int32, which on some (all?) 32 bit platforms is a long (7) rather than an int. I think this is a bug and it will cause problems with numpy 1.6.
You mean ndimage I guess. I don't really understand your explanation, if that's the case then those ndimage tests should have been failing before, right?
If this needs to be fixed in scipy then we need an RC3. I would like to get that out by the 12th or so if possible.
I think it needs to be fixed as it is a bug. The problem is that ndimage checks types, but only by Int16, Int32, etc. On 32 bit systems and windows Int32 is a long (type# 7) and integers (type# 5), are treated as unrecognized types even though they are the same thing. Ndimage should accept both types.
I'm still a bit puzzled about why it's only failing after the recent
changes in numpy master.
Sounds like it's a bug, but it's not a regression and we are at rc2 right now. Do you (or does someone else) have time to fix it in the next week or so? The other option is to fix it in 0.9.1, which can come out at the same time as numpy 1.6.0.
Sorry, I hadn't noticed that there's already a patch attached to http://projects.scipy.org/numpy/ticket/1724 If you could review and commit it, that would be great though. Ralf
On Tue, Feb 1, 2011 at 10:36 PM, Ralf Gommers <ralf.gommers@googlemail.com>wrote:
On Wed, Feb 2, 2011 at 1:11 PM, Ralf Gommers <ralf.gommers@googlemail.com>wrote:
On Wed, Feb 2, 2011 at 12:57 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Tue, Feb 1, 2011 at 9:04 PM, Ralf Gommers < ralf.gommers@googlemail.com> wrote:
On Mon, Jan 31, 2011 at 11:01 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
I think there should be a fix for ndarray also, the problem with type 5 (int) not being recognized is that it checks for Int32, which on some (all?) 32 bit platforms is a long (7) rather than an int. I think this is a bug and it will cause problems with numpy 1.6.
You mean ndimage I guess. I don't really understand your explanation, if that's the case then those ndimage tests should have been failing before, right?
If this needs to be fixed in scipy then we need an RC3. I would like to get that out by the 12th or so if possible.
I think it needs to be fixed as it is a bug. The problem is that ndimage checks types, but only by Int16, Int32, etc. On 32 bit systems and windows Int32 is a long (type# 7) and integers (type# 5), are treated as unrecognized types even though they are the same thing. Ndimage should accept both types.
I'm still a bit puzzled about why it's only failing after the recent
changes in numpy master.
Sounds like it's a bug, but it's not a regression and we are at rc2 right now. Do you (or does someone else) have time to fix it in the next week or so? The other option is to fix it in 0.9.1, which can come out at the same time as numpy 1.6.0.
Sorry, I hadn't noticed that there's already a patch attached to http://projects.scipy.org/numpy/ticket/1724 If you could review and commit it, that would be great though.
I've gone ahead and committed it to the trunk in r7120, the decision to backport it or not I'll leave to you. It could probably be made a bit more bombproof with respect to integers but I think it covers all the platforms I'm familiar with. We can make things more determinate for numpy 2.0. Chuck
On Wed, Feb 2, 2011 at 10:57 AM, Charles R Harris <charlesr.harris@gmail.com
wrote:
On Tue, Feb 1, 2011 at 10:36 PM, Ralf Gommers <ralf.gommers@googlemail.com
wrote:
On Wed, Feb 2, 2011 at 1:11 PM, Ralf Gommers <ralf.gommers@googlemail.com
wrote:
On Wed, Feb 2, 2011 at 12:57 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
On Tue, Feb 1, 2011 at 9:04 PM, Ralf Gommers < ralf.gommers@googlemail.com> wrote:
On Mon, Jan 31, 2011 at 11:01 PM, Charles R Harris < charlesr.harris@gmail.com> wrote:
I think there should be a fix for ndarray also, the problem with type 5 (int) not being recognized is that it checks for Int32, which on some (all?) 32 bit platforms is a long (7) rather than an int. I think this is a bug and it will cause problems with numpy 1.6.
You mean ndimage I guess. I don't really understand your explanation, if that's the case then those ndimage tests should have been failing before, right?
If this needs to be fixed in scipy then we need an RC3. I would like to get that out by the 12th or so if possible.
I think it needs to be fixed as it is a bug. The problem is that ndimage checks types, but only by Int16, Int32, etc. On 32 bit systems and windows Int32 is a long (type# 7) and integers (type# 5), are treated as unrecognized types even though they are the same thing. Ndimage should accept both types.
I'm still a bit puzzled about why it's only failing after the recent
changes in numpy master.
Sounds like it's a bug, but it's not a regression and we are at rc2 right now. Do you (or does someone else) have time to fix it in the next week or so? The other option is to fix it in 0.9.1, which can come out at the same time as numpy 1.6.0.
Sorry, I hadn't noticed that there's already a patch attached to http://projects.scipy.org/numpy/ticket/1724 If you could review and commit it, that would be great though.
I've gone ahead and committed it to the trunk in r7120, the decision to backport it or not I'll leave to you. It could probably be made a bit more bombproof with respect to integers but I think it covers all the platforms I'm familiar with. We can make things more determinate for numpy 2.0.
In particular, the patch should probably be tested on 64 bit SPARC with the sun compilers and maybe on ALPHA and HPPA also. Anyone? Chuck
On 1/30/2011 5:50 PM, Ralf Gommers wrote:
Hi,
I am pleased to announce the availability of the second release candidate of SciPy 0.9.0. This will be the first SciPy release to include support for Python 3 (all modules except scipy.weave), as well as for Python 2.7.
Due to the Sourceforge outage I am not able to put binaries on the normal download site right now, that will probably only happen in a week or so. If you want to try the RC now please build from svn, and report any issues.
Changes since release candidate 1: - fixes for build problems with MSVC + MKL (#1210, #1376) - fix pilutil test to work with numpy master branch - fix constants.codata to be backwards-compatible
Enjoy, Ralf
I tested msvc9/ifort/MKL builds of scipy 0.9 rc2 with Python 2.6, 2.7, 3.1 and 3.2 on win32 and win-amd64. Besides the known problems with the MKL builds (<http://projects.scipy.org/scipy/ticket/1238>, <http://projects.scipy.org/scipy/ticket/1233>, <http://projects.scipy.org/scipy/ticket/678>), the following tests fail on win32 only: ====================================================================== FAIL: test_linesearch.TestLineSearch.test_line_search_armijo ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File "X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py",line 201, in test_line_search_armijo assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 1.5675494536393939 DESIRED: 1.5675494536393932 ====================================================================== FAIL: test_linesearch.TestLineSearch.test_line_search_wolfe1 ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File "X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py", line 164, in test_line_search_wolfe1 assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 19.185353513927268 DESIRED: 19.185353513927272 ====================================================================== FAIL: test_linesearch.TestLineSearch.test_line_search_wolfe2 ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File "X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py",line 184, in test_line_search_wolfe2 assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 19.185353513927272 DESIRED: 19.185353513927268 ====================================================================== FAIL: Powell (direction set) optimization routine ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\scipy\optimize\tests\test_optimize.py", line 123, in test_powell assert_(self.funccalls == 116, self.funccalls) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 34, in assert_ raise AssertionError(msg) AssertionError: 128 -- Christoph
On Mon, Jan 31, 2011 at 11:56 PM, Christoph Gohlke <cgohlke@uci.edu> wrote:
On 1/30/2011 5:50 PM, Ralf Gommers wrote:
Hi,
I am pleased to announce the availability of the second release candidate of SciPy 0.9.0. This will be the first SciPy release to include support for Python 3 (all modules except scipy.weave), as well as for Python 2.7.
Due to the Sourceforge outage I am not able to put binaries on the normal download site right now, that will probably only happen in a week or so. If you want to try the RC now please build from svn, and report any issues.
Changes since release candidate 1: - fixes for build problems with MSVC + MKL (#1210, #1376) - fix pilutil test to work with numpy master branch - fix constants.codata to be backwards-compatible
Enjoy, Ralf
I tested msvc9/ifort/MKL builds of scipy 0.9 rc2 with Python 2.6, 2.7, 3.1 and 3.2 on win32 and win-amd64. Besides the known problems with the MKL builds (<http://projects.scipy.org/scipy/ticket/1238>, <http://projects.scipy.org/scipy/ticket/1233>, <http://projects.scipy.org/scipy/ticket/678>), the following tests fail on win32 only:
Tor the first three failures I can adjust the test precision to make the tests pass. For the Powell failure, what are the actual number of funccalls? That type of assertion doesn't work too well, should be <= 116 instead of ==. Ralf
====================================================================== FAIL: test_linesearch.TestLineSearch.test_line_search_armijo ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File
"X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py",line 201, in test_line_search_armijo assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 1.5675494536393939 DESIRED: 1.5675494536393932
====================================================================== FAIL: test_linesearch.TestLineSearch.test_line_search_wolfe1 ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File "X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py", line 164, in test_line_search_wolfe1 assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 19.185353513927268 DESIRED: 19.185353513927272
====================================================================== FAIL: test_linesearch.TestLineSearch.test_line_search_wolfe2 ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File
"X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py",line 184, in test_line_search_wolfe2 assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 19.185353513927272 DESIRED: 19.185353513927268
====================================================================== FAIL: Powell (direction set) optimization routine ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\scipy\optimize\tests\test_optimize.py", line 123, in test_powell assert_(self.funccalls == 116, self.funccalls) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 34, in assert_ raise AssertionError(msg) AssertionError: 128
-- Christoph _______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On 2/1/2011 8:07 PM, Ralf Gommers wrote:
On Mon, Jan 31, 2011 at 11:56 PM, Christoph Gohlke <cgohlke@uci.edu <mailto:cgohlke@uci.edu>> wrote:
On 1/30/2011 5:50 PM, Ralf Gommers wrote: > Hi, > > I am pleased to announce the availability of the second release > candidate of SciPy 0.9.0. This will be the first SciPy release to > include support for Python 3 (all modules except scipy.weave), as well > as for Python 2.7. > > Due to the Sourceforge outage I am not able to put binaries on the > normal download site right now, that will probably only happen in a week > or so. If you want to try the RC now please build from svn, and report > any issues. > > Changes since release candidate 1: > - fixes for build problems with MSVC + MKL (#1210, #1376) > - fix pilutil test to work with numpy master branch > - fix constants.codata to be backwards-compatible > > Enjoy, > Ralf >
I tested msvc9/ifort/MKL builds of scipy 0.9 rc2 with Python 2.6, 2.7, 3.1 and 3.2 on win32 and win-amd64. Besides the known problems with the MKL builds (<http://projects.scipy.org/scipy/ticket/1238>, <http://projects.scipy.org/scipy/ticket/1233>, <http://projects.scipy.org/scipy/ticket/678>), the following tests fail on win32 only:
Tor the first three failures I can adjust the test precision to make the tests pass. For the Powell failure, what are the actual number of funccalls? That type of assertion doesn't work too well, should be <= 116 instead of ==. Ralf
I ran the test in a loop: most of the times the number of funccalls is 116, often 117, and sometimes as large as 128. Not sure why this test only fails on win32 and not win-amd64. Christoph
====================================================================== FAIL: test_linesearch.TestLineSearch.test_line_search_armijo ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File "X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py",line 201, in test_line_search_armijo assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 1.5675494536393939 DESIRED: 1.5675494536393932
====================================================================== FAIL: test_linesearch.TestLineSearch.test_line_search_wolfe1 ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File "X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py", line 164, in test_line_search_wolfe1 assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 19.185353513927268 DESIRED: 19.185353513927272
====================================================================== FAIL: test_linesearch.TestLineSearch.test_line_search_wolfe2 ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File "X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py",line 184, in test_line_search_wolfe2 assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 19.185353513927272 DESIRED: 19.185353513927268
====================================================================== FAIL: Powell (direction set) optimization routine ---------------------------------------------------------------------- Traceback (most recent call last): File "X:\Python27\lib\site-packages\scipy\optimize\tests\test_optimize.py", line 123, in test_powell assert_(self.funccalls == 116, self.funccalls) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 34, in assert_ raise AssertionError(msg) AssertionError: 128
-- Christoph _______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org <mailto:SciPy-Dev@scipy.org> http://mail.scipy.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Wed, Feb 2, 2011 at 12:34 PM, Christoph Gohlke <cgohlke@uci.edu> wrote:
On 2/1/2011 8:07 PM, Ralf Gommers wrote:
On Mon, Jan 31, 2011 at 11:56 PM, Christoph Gohlke <cgohlke@uci.edu <mailto:cgohlke@uci.edu>> wrote:
On 1/30/2011 5:50 PM, Ralf Gommers wrote: > Hi, > > I am pleased to announce the availability of the second release > candidate of SciPy 0.9.0. This will be the first SciPy release to > include support for Python 3 (all modules except scipy.weave), as
well
> as for Python 2.7. > > Due to the Sourceforge outage I am not able to put binaries on the > normal download site right now, that will probably only happen in a week > or so. If you want to try the RC now please build from svn, and
report
> any issues. > > Changes since release candidate 1: > - fixes for build problems with MSVC + MKL (#1210, #1376) > - fix pilutil test to work with numpy master branch > - fix constants.codata to be backwards-compatible > > Enjoy, > Ralf >
I tested msvc9/ifort/MKL builds of scipy 0.9 rc2 with Python 2.6,
2.7,
3.1 and 3.2 on win32 and win-amd64. Besides the known problems with
the
MKL builds (<http://projects.scipy.org/scipy/ticket/1238>, <http://projects.scipy.org/scipy/ticket/1233>, <http://projects.scipy.org/scipy/ticket/678>), the following tests
fail
on win32 only:
Tor the first three failures I can adjust the test precision to make the tests pass. For the Powell failure, what are the actual number of funccalls? That type of assertion doesn't work too well, should be <= 116 instead of ==. Ralf
I ran the test in a loop: most of the times the number of funccalls is 116, often 117, and sometimes as large as 128. Not sure why this test only fails on win32 and not win-amd64.
It looks like an actual problem then, and not just a difference between
platforms. I saw something like that recently in a Wolfe line search test too, the problem was floating point ==0.0 comparison. Can you open a ticket? Ralf
======================================================================
FAIL: test_linesearch.TestLineSearch.test_line_search_armijo
----------------------------------------------------------------------
Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File
"X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py",line
201, in test_line_search_armijo assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 1.5675494536393939 DESIRED: 1.5675494536393932
======================================================================
FAIL: test_linesearch.TestLineSearch.test_line_search_wolfe1
----------------------------------------------------------------------
Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File
"X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py",
line 164, in test_line_search_wolfe1 assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 19.185353513927268 DESIRED: 19.185353513927272
======================================================================
FAIL: test_linesearch.TestLineSearch.test_line_search_wolfe2
----------------------------------------------------------------------
Traceback (most recent call last): File "X:\Python27\lib\site-packages\nose\case.py", line 187, in runTest self.test(*self.arg) File
"X:\Python27\lib\site-packages\scipy\optimize\tests\test_linesearch.py",line
184, in test_line_search_wolfe2 assert_equal(fv, f(x + s*p)) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line 313, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 19.185353513927272 DESIRED: 19.185353513927268
======================================================================
FAIL: Powell (direction set) optimization routine
----------------------------------------------------------------------
Traceback (most recent call last): File
"X:\Python27\lib\site-packages\scipy\optimize\tests\test_optimize.py",
line 123, in test_powell assert_(self.funccalls == 116, self.funccalls) File "X:\Python27\lib\site-packages\numpy\testing\utils.py", line
34,
in assert_ raise AssertionError(msg) AssertionError: 128
-- Christoph _______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org <mailto:SciPy-Dev@scipy.org> http://mail.scipy.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
Wed, 02 Feb 2011 13:17:03 +0800, Ralf Gommers wrote: [clip: test_optimize.TestOptimize.test_powell]
I ran the test in a loop: most of the times the number of funccalls is 116, often 117, and sometimes as large as 128. Not sure why this test only fails on win32 and not win-amd64.
It looks like an actual problem then, and not just a difference between platforms. I saw something like that recently in a Wolfe line search test too, the problem was floating point == 0.0 comparison.
Can you open a ticket?
The implementation of fmin_powell has not changed since Scipy 0.7. I'd imagine that the exact iteration count might be somewhat sensitive to numerical noise (due to the endgame of optimization). So, probably, it is mostly an issue with the test. When I wrote those tests, I hoped (naively) that FP arithmetic would be reproducible across machines, which turns out not to be true. Now it seems that it is not exactly reproducible even across multiple iterations on the same machine. Frankly, I do not understand where the random variation comes from --- the test does not use any random numbers etc. The only explanation I can imagine that something (MKL?) on Windows platforms does not perform exactly reproducible floating-point arithmetic. @Cristoph: Have you been able to reproduce random variation in results from numpy.dot? If yes, does the same variations occur in results from numpy.core.multiarray.dot (which never uses BLAS)? I would be much happier if we could point a finger towards MKL as the culprit for non- reproducible FP results. Pauli
Wed, 02 Feb 2011 12:53:11 +0000, Pauli Virtanen wrote: [clip]
@Cristoph: Have you been able to reproduce random variation in results from numpy.dot? If yes, does the same variations occur in results from numpy.core.multiarray.dot (which never uses BLAS)? I would be much happier if we could point a finger towards MKL as the culprit for non- reproducible FP results.
OK, seems to be MKL http://software.intel.com/sites/products/documentation/hpc/mkl/lin/ MKL_UG_coding_tips/Aligning_Data_for_Numerical_Stability.htm I wouldn't be too surprised if data alignment and threading also had a similar effect with ATLAS. The correct solution is probably just to bump the test tolerances so that they pass with MKL. -- Pauli Virtanen
On Wed, Feb 2, 2011 at 9:31 AM, Pauli Virtanen <pav@iki.fi> wrote:
Wed, 02 Feb 2011 12:53:11 +0000, Pauli Virtanen wrote: [clip]
@Cristoph: Have you been able to reproduce random variation in results from numpy.dot? If yes, does the same variations occur in results from numpy.core.multiarray.dot (which never uses BLAS)? I would be much happier if we could point a finger towards MKL as the culprit for non- reproducible FP results.
OK, seems to be MKL
http://software.intel.com/sites/products/documentation/hpc/mkl/lin/ MKL_UG_coding_tips/Aligning_Data_for_Numerical_Stability.htm
I wouldn't be too surprised if data alignment and threading also had a similar effect with ATLAS.
From this it looks like the problem is more general, but until now we have seen it only on Win32.
The correct solution is probably just to bump the test tolerances so that they pass with MKL.
It's good to know it's not anything "serious". Is there a good place in the docs to warn about this? Josef
-- Pauli Virtanen
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
Wed, 02 Feb 2011 09:43:04 -0500, josef.pktd wrote: [clip]
From this it looks like the problem is more general, but until now we have seen it only on Win32.
If the matter is data alignment, perhaps this is because of memory allocators or the BLAS library functioning differently on different platforms. To get a definite answer if the issue is data alignment, one can perhaps check if the different results are correlated with input_array.__array_interface__['data'][0] % 16 (assuming no copies are made inside dot()). -- Pauli Virtanen
On 2/2/2011 6:50 AM, Pauli Virtanen wrote:
Wed, 02 Feb 2011 09:43:04 -0500, josef.pktd wrote: [clip]
From this it looks like the problem is more general, but until now we have seen it only on Win32.
If the matter is data alignment, perhaps this is because of memory allocators or the BLAS library functioning differently on different platforms.
To get a definite answer if the issue is data alignment, one can perhaps check if the different results are correlated with
input_array.__array_interface__['data'][0] % 16
(assuming no copies are made inside dot()).
Yes, the 'K' array used in the test is not always aligned optimally. If K is aligned, the number of iterations is as expected (116) and the test passes. If K is not aligned, the number of iterations is > 116. It does not matter to use numpy.core.multiarray.dot instead of numpy.dot in the func() function. The 'params' array returned by optimize.fmin_powell() varies when K is unaligned but it always passes the 'abs(func(params) - func(solution)) < 1e-6' test so it should be safe to increase the test tolerance (self.funccalls <= 128). Christoph
On Wed, Feb 2, 2011 at 9:50 AM, Pauli Virtanen <pav@iki.fi> wrote:
Wed, 02 Feb 2011 09:43:04 -0500, josef.pktd wrote: [clip]
From this it looks like the problem is more general, but until now we have seen it only on Win32.
If the matter is data alignment, perhaps this is because of memory allocators or the BLAS library functioning differently on different platforms.
To get a definite answer if the issue is data alignment, one can perhaps check if the different results are correlated with
input_array.__array_interface__['data'][0] % 16
(assuming no copies are made inside dot()).
alignment alternates, same script as on numpy mailinglist (numpy.dot) with additional print print (y[0], m.__array_interface__['data'][0] % 16, x.__array_interface__['data'][0] % 16) using lapack (array([ 5.00486230e-06]), 8, 0) (array([ 5.00486408e-06]), 0, 0) (array([ 5.00486230e-06]), 8, 0) (array([ 5.00486408e-06]), 0, 0) (array([ 5.00486230e-06]), 8, 0) (array([ 5.00486408e-06]), 0, 0) (array([ 5.00486230e-06]), 8, 0) (array([ 5.00486408e-06]), 0, 0) (array([ 5.00486230e-06]), 8, 0) (array([ 5.00486408e-06]), 0, 0) in some runs alignment doesn't change in iterations using lapack (array([ 5.00486230e-06]), 8, 8) (array([ 5.00486230e-06]), 8, 8) (array([ 5.00486230e-06]), 8, 8) (array([ 5.00486230e-06]), 8, 8) (array([ 5.00486230e-06]), 8, 8) (array([ 5.00486230e-06]), 8, 8) (array([ 5.00486230e-06]), 8, 8) (array([ 5.00486230e-06]), 8, 8) (array([ 5.00486230e-06]), 8, 8) (array([ 5.00486230e-06]), 8, 8) Josef
-- Pauli Virtanen
_______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
participants (5)
-
Charles R Harris -
Christoph Gohlke -
josef.pktd＠gmail.com -
Pauli Virtanen -
Ralf Gommers