Hi Jarrod, Just curious. Is there a new deadline for scipy 0.7 ? Cheers, Nils
On Sat, Sep 27, 2008 at 9:14 AM, Nils Wagner <nwagner@iam.uni-stuttgart.de> wrote:
Just curious. Is there a new deadline for scipy 0.7 ?
I just haven't had any free time. My plan at this point is to release an alpha first with binaries. Since the trunk has had so many changes, I would like to see more testing before releasing a beta. If anyone has time and interest in helping, it would make a huge difference in getting this release out. If someone could hunt down the and fix the test failures and clean up all the deprecation warnings, we would be much closer to getting this release out the door. Thanks, -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/
Jarrod Millman wrote:
I just haven't had any free time. My plan at this point is to release an alpha first with binaries. Since the trunk has had so many changes, I would like to see more testing before releasing a beta.
Before an alpha, there are several things which I consider a blocker for a release and would like to fix (and seeing fixed): - http://projects.scipy.org/scipy/scipy/ticket/244 -> fft gives wrong results with some axes parameters - http://projects.scipy.org/scipy/scipy/ticket/691 -> valgrind warnings - http://projects.scipy.org/scipy/scipy/ticket/688 -> fftpack revamp - http://projects.scipy.org/scipy/scipy/ticket/725 -> arpack crashes/gives wrong result on Mac OS X For windows binaries: taking the same approach as for numpy would work, but would give a big installer (~40 Mb). I don't see any other approach, except just linking with netlib (having a net installer sounds like a support nightmare with all the proxy, firewall, etc...) cheers, David
On Tue, 30 Sep 2008 11:52:32 +0900 David Cournapeau <david@ar.media.kyoto-u.ac.jp> wrote:
Jarrod Millman wrote:
I just haven't had any free time. My plan at this point is to release an alpha first with binaries. Since the trunk has had so many changes, I would like to see more testing before releasing a beta.
Before an alpha, there are several things which I consider a blocker for a release and would like to fix (and seeing fixed):
- http://projects.scipy.org/scipy/scipy/ticket/244 -> fft gives wrong results with some axes parameters - http://projects.scipy.org/scipy/scipy/ticket/691 -> valgrind warnings - http://projects.scipy.org/scipy/scipy/ticket/688 -> fftpack revamp - http://projects.scipy.org/scipy/scipy/ticket/725 -> arpack crashes/gives wrong result on Mac OS X
For windows binaries: taking the same approach as for numpy would work, but would give a big installer (~40 Mb). I don't see any other approach, except just linking with netlib (having a net installer sounds like a support nightmare with all the proxy, firewall, etc...)
cheers,
David _______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
Hi all, IMHO, http://projects.scipy.org/scipy/scipy/ticket/709 should be resolved, too. Nils
Tue, 30 Sep 2008 19:56:58 +0200, Nils Wagner wrote:
IMHO,
http://projects.scipy.org/scipy/scipy/ticket/709
should be resolved, too.
Short summary about this one: it's about a special matrix pair for which LAPACK's generalized eigenvalue problem solver DGGEV returns bogus output, losing one eigenvalue from a complex eigenpair. (The eigenvalue problem should nonetheless be well-defined, AFAIK, this looks like a bug in LAPACK.) The question here is what we should/can do: 1. Raise LinAlgError if we detect this condition. 2. Try to repair the returned data by filling in the other eigenvalue of the pair, as we know all complex eigenvalues come in pairs. 3. Try to return as much as we can make sense of, but put NaN in place of the missing eigenvalue. Details: Instead of expected complex eigenvalue pair (a_R +- i a_I)/beta, the LAPACK routine instead returns the values a_R = whatever a_I = [0., -2.18645248e-16j] beta = [0., 7.95804395e-17] The correct result would have been something like a_R = whatever a_I = [+2.18645248e-16j, -2.18645248e-16j] beta = [7.95804395e-17, 7.95804395e-17] but one eigenvalue of the pair was lost, apparently due to rounding etc. Eigenvector data corresponding to zeros in a_I are interpreted differently than for positive entries, so the zero in the wrong place messed up the Scipy routine. We could detect this and raise an error (1), substitute in the missing eigenvalue based on the correctly calculated one (2), or leave the missing eigenvalue as corrupted and return only the correct one (3). -- Pauli Virtanen
On Tue, 30 Sep 2008 18:36:12 +0000 (UTC) Pauli Virtanen <pav@iki.fi> wrote:
Tue, 30 Sep 2008 19:56:58 +0200, Nils Wagner wrote:
IMHO,
http://projects.scipy.org/scipy/scipy/ticket/709
should be resolved, too.
Short summary about this one: it's about a special matrix pair for which LAPACK's generalized eigenvalue problem solver DGGEV returns bogus output, losing one eigenvalue from a complex eigenpair. (The eigenvalue problem should nonetheless be well-defined, AFAIK, this looks like a bug in LAPACK.)
In that case the LAPACK developers should be informed as soon as possible. Did you check a FORTRAN implementation of the test case ? Can someone run the test (2dof.py) in Matlab, Octave, Scilab ?
The question here is what we should/can do:
1. Raise LinAlgError if we detect this condition.
2. Try to repair the returned data by filling in the other eigenvalue of the pair, as we know all complex eigenvalues come in pairs.
Well, this holds for real matrices but what could be done in case of complex matrices ? Nils
3. Try to return as much as we can make sense of, but put NaN in place of the missing eigenvalue.
Details:
Instead of expected complex eigenvalue pair (a_R +- i a_I)/beta, the LAPACK routine instead returns the values
a_R = whatever a_I = [0., -2.18645248e-16j] beta = [0., 7.95804395e-17]
The correct result would have been something like
a_R = whatever a_I = [+2.18645248e-16j, -2.18645248e-16j] beta = [7.95804395e-17, 7.95804395e-17]
but one eigenvalue of the pair was lost, apparently due to rounding etc.
Eigenvector data corresponding to zeros in a_I are interpreted differently than for positive entries, so the zero in the wrong place messed up the Scipy routine. We could detect this and raise an error (1), substitute in the missing eigenvalue based on the correctly calculated one (2), or leave the missing eigenvalue as corrupted and return only the correct one (3).
-- Pauli Virtanen
_______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.org http://projects.scipy.org/mailman/listinfo/scipy-dev
Tue, 30 Sep 2008 21:21:38 +0200, Nils Wagner wrote:
On Tue, 30 Sep 2008 18:36:12 +0000 (UTC) Pauli Virtanen <pav@iki.fi> wrote:
Tue, 30 Sep 2008 19:56:58 +0200, Nils Wagner wrote:
IMHO,
http://projects.scipy.org/scipy/scipy/ticket/709
should be resolved, too.
Short summary about this one: it's about a special matrix pair for which LAPACK's generalized eigenvalue problem solver DGGEV returns bogus output, losing one eigenvalue from a complex eigenpair. (The eigenvalue problem should nonetheless be well-defined, AFAIK, this looks like a bug in LAPACK.)
In that case the LAPACK developers should be informed as soon as possible. Did you check a FORTRAN implementation of the test case ?
I didn't contact LAPACK devs yet. I'll try to see if I can reduce this to a simple Fortran testcase. But to my understanding, LAPACK's DGEEV has also some other known issues, see eg. http://icl.cs.utk.edu/lapack-forum/viewtopic.php? f=2&t=317&p=1060&hilit=dggev
Can someone run the test (2dof.py) in Matlab, Octave, Scilab ?
I got similar suspect result from Octave's QZ when I tried it (Octave dropped one of the eigenvalues, too). Matlab AFAIK does not use Lapack's DGGEV, so the testcase probably will not fail in the same way with it.
The question here is what we should/can do:
1. Raise LinAlgError if we detect this condition.
2. Try to repair the returned data by filling in the other eigenvalue of the pair, as we know all complex eigenvalues come in pairs.
Well, this holds for real matrices but what could be done in case of complex matrices ?
For complex matrices the eigenvectors are returned by LAPACK as complex data directly (and don't need to be separately constructed from real data, as in the real case), so they are simply passed throught by Scipy. If the results are wrong, we probably can't do anything to fix or detect this. -- Pauli Virtanen
On Tue, 30 Sep 2008 19:43:10 +0000 (UTC) Pauli Virtanen <pav@iki.fi> wrote:
Tue, 30 Sep 2008 21:21:38 +0200, Nils Wagner wrote:
On Tue, 30 Sep 2008 18:36:12 +0000 (UTC) Pauli Virtanen <pav@iki.fi> wrote:
Tue, 30 Sep 2008 19:56:58 +0200, Nils Wagner wrote:
IMHO,
http://projects.scipy.org/scipy/scipy/ticket/709
should be resolved, too.
Short summary about this one: it's about a special matrix pair for which LAPACK's generalized eigenvalue problem solver DGGEV returns bogus output, losing one eigenvalue from a complex eigenpair. (The eigenvalue problem should nonetheless be well-defined, AFAIK, this looks like a bug in LAPACK.)
In that case the LAPACK developers should be informed as soon as possible. Did you check a FORTRAN implementation of the test case ?
I didn't contact LAPACK devs yet. I'll try to see if I can reduce this to a simple Fortran testcase.
But to my understanding, LAPACK's DGEEV has also some other known issues, see eg. http://icl.cs.utk.edu/lapack-forum/viewtopic.php? f=2&t=317&p=1060&hilit=dggev
Can someone run the test (2dof.py) in Matlab, Octave, Scilab ?
I got similar suspect result from Octave's QZ when I tried it (Octave dropped one of the eigenvalues, too).
Matlab AFAIK does not use Lapack's DGGEV, so the testcase probably will not fail in the same way with it.
The question here is what we should/can do:
1. Raise LinAlgError if we detect this condition.
2. Try to repair the returned data by filling in the other eigenvalue of the pair, as we know all complex eigenvalues come in pairs.
Well, this holds for real matrices but what could be done in case of complex matrices ?
For complex matrices the eigenvectors are returned by LAPACK as complex data directly (and don't need to be separately constructed from real data, as in the real case), so they are simply passed throught by Scipy. If the results are wrong, we probably can't do anything to fix or detect this.
-- Pauli Virtanen
Hi Pauli, Please find attached an u n t e s t e d FORTRAN implementation of 2dof.py. Cheers, Nils
Tue, 30 Sep 2008 22:23:49 +0200, Nils Wagner wrote:
Please find attached an u n t e s t e d FORTRAN implementation of 2dof.py.
Tested pure-Fortran testcase is here: http://scipy.org/scipy/scipy/attachment/ticket/709/2dof.f?format=raw Could you check that it reproduces the problem for you, and list the problematic values of Omega on your platform? So far, I've seen the following pattern: platform 1 (32-bit x86 AMD, Ubuntu Interpid, Lapack 3.1.1) Omega = 25/99, 110/99, 155/99, 265/99 platform 2 (64-bit x86 Intel, Debian etch, Lapack 3.0) Omega = 185/99 -- Pauli Virtanen
On Mon, Sep 29, 2008 at 9:55 AM, Jarrod Millman <millman@berkeley.edu> wrote:
Just curious. Is there a new deadline for scipy 0.7 ?
If anyone has time and interest in helping, it would make a huge difference in getting this release out. If someone could hunt down the and fix the test failures and clean up all the deprecation warnings, we would be much closer to getting this release out the door.
I'll try to fix some of the deprecation warnings and/or look at failing tests this week or over the weekend. Alan
On Tue, Sep 30, 2008 at 9:37 PM, Alan McIntyre <alan.mcintyre@gmail.com> wrote:
On Mon, Sep 29, 2008 at 9:55 AM, Jarrod Millman <millman@berkeley.edu> wrote:
Just curious. Is there a new deadline for scipy 0.7 ?
If anyone has time and interest in helping, it would make a huge difference in getting this release out. If someone could hunt down the and fix the test failures and clean up all the deprecation warnings, we would be much closer to getting this release out the door.
I'll try to fix some of the deprecation warnings and/or look at failing tests this week or over the weekend.
Thanks! The sooner we can get some of these issues resolved the sooner we can get a release out. -- Jarrod Millman Computational Infrastructure for Research Labs 10 Giannini Hall, UC Berkeley phone: 510.643.4014 http://cirl.berkeley.edu/
Hi, I think that my fixes to scipy.stats.distributions in http://scipy.org/scipy/scipy/ticket/745 are now good enough that they can go into 0.7, after some review by numpy experts which I am not. As far as I can see, non of my fixes changes anything in the existing functionality, but fixes several methods that throw exceptions in the current trunk, and it corrects incorrect calculations in some places. I provided basic tests. The test results now look pretty good, compared to before. This is still not a complete test suite for scipy.stats.distributions, but in my opinion, a big enhancement to the current tests. Josef
participants (7)
-
Alan McIntyre -
David Cournapeau -
Jarrod Millman -
josef.pktd＠gmail.com -
Nils Wagner -
Ondrej Certik -
Pauli Virtanen