Warnings raised (from fit in scipy.stats)
Since the raising of warning behavior has been changed (I believe), I have been running into a lot of warnings in my code when say I do something like In [120]: from scipy import stats In [121]: y = [-45, -3, 1, 0, 1, 3] In [122]: v = stats.norm.pdf(y)/stats.norm.cdf(y) Warning: invalid value encountered in divide Sometimes, this is useful to know. Sometimes, though, it's very disturbing when it's encountered in some kind of iteration or optimization. I have been using numpy.clip to get around this in my own code, but when it's buried a bit deeper, it's not quite so simple. Take this example. In [123]: import numpy as np In [124]: np.random.seed(12345) In [125]: B = 6.0 In [126]: x = np.random.exponential(scale=B, size=5000) In [127]: from scipy.stats import expon In [128]: expon.fit(x) <dozens of warnings clipped> Out[128]: (0.21874043533906118, 5.7122829778172939) The fit is achieved by fmin (as far as I know, since disp=0 in the rv_continuous.fit...), but there are a number of warnings emitted. Is there any middle ground to be had in these type of situations via context management perhaps? Should I file a ticket? Skipper
On Fri, Jun 11, 2010 at 12:45 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
Since the raising of warning behavior has been changed (I believe), I have been running into a lot of warnings in my code when say I do something like
In [120]: from scipy import stats
In [121]: y = [-45, -3, 1, 0, 1, 3]
In [122]: v = stats.norm.pdf(y)/stats.norm.cdf(y) Warning: invalid value encountered in divide
Sometimes, this is useful to know. Sometimes, though, it's very disturbing when it's encountered in some kind of iteration or optimization. I have been using numpy.clip to get around this in my own code, but when it's buried a bit deeper, it's not quite so simple.
Take this example.
In [123]: import numpy as np
In [124]: np.random.seed(12345)
In [125]: B = 6.0
In [126]: x = np.random.exponential(scale=B, size=5000)
In [127]: from scipy.stats import expon
In [128]: expon.fit(x)
<dozens of warnings clipped>
Out[128]: (0.21874043533906118, 5.7122829778172939)
The fit is achieved by fmin (as far as I know, since disp=0 in the rv_continuous.fit...), but there are a number of warnings emitted. Is there any middle ground to be had in these type of situations via context management perhaps?
Should I file a ticket?
Which numpy scipy versions are you using? I don't get any warning with the first example. (numpy 1.4.0) (I cannot run the second example because I have a scipy revision with a broken fit() method) I don't think wrapping functions/methods to turn off warnings is a good option. (many of them are in inner loops for example for random number generation) Josef
Skipper _______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Fri, Jun 11, 2010 at 1:07 PM, <josef.pktd@gmail.com> wrote:
On Fri, Jun 11, 2010 at 12:45 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
Since the raising of warning behavior has been changed (I believe), I have been running into a lot of warnings in my code when say I do something like
In [120]: from scipy import stats
In [121]: y = [-45, -3, 1, 0, 1, 3]
In [122]: v = stats.norm.pdf(y)/stats.norm.cdf(y) Warning: invalid value encountered in divide
Sometimes, this is useful to know. Sometimes, though, it's very disturbing when it's encountered in some kind of iteration or optimization. I have been using numpy.clip to get around this in my own code, but when it's buried a bit deeper, it's not quite so simple.
Take this example.
In [123]: import numpy as np
In [124]: np.random.seed(12345)
In [125]: B = 6.0
In [126]: x = np.random.exponential(scale=B, size=5000)
In [127]: from scipy.stats import expon
In [128]: expon.fit(x)
<dozens of warnings clipped>
Out[128]: (0.21874043533906118, 5.7122829778172939)
The fit is achieved by fmin (as far as I know, since disp=0 in the rv_continuous.fit...), but there are a number of warnings emitted. Is there any middle ground to be had in these type of situations via context management perhaps?
Should I file a ticket?
Which numpy scipy versions are you using?
Numpy '2.0.0.dev8417' Scipy '0.9.0.dev6447'
I don't get any warning with the first example. (numpy 1.4.0) (I cannot run the second example because I have a scipy revision with a broken fit() method)
I don't think wrapping functions/methods to turn off warnings is a good option. (many of them are in inner loops for example for random number generation)
Granted I haven't looked too much into the details of the warnings context manager other than using some toy examples once or twice, but if you could just suppress them for when the solver is called within a function/method then this would do the trick (at least for the ones I have been running into, mostly to do with fitting like this or with maximum likelihood). Skipper
On Fri, Jun 11, 2010 at 11:34 AM, Skipper Seabold <jsseabold@gmail.com> wrote:
On Fri, Jun 11, 2010 at 1:07 PM, <josef.pktd@gmail.com> wrote:
On Fri, Jun 11, 2010 at 12:45 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
Since the raising of warning behavior has been changed (I believe), I have been running into a lot of warnings in my code when say I do something like
In [120]: from scipy import stats
In [121]: y = [-45, -3, 1, 0, 1, 3]
In [122]: v = stats.norm.pdf(y)/stats.norm.cdf(y) Warning: invalid value encountered in divide
Sometimes, this is useful to know. Sometimes, though, it's very disturbing when it's encountered in some kind of iteration or optimization. I have been using numpy.clip to get around this in my own code, but when it's buried a bit deeper, it's not quite so simple.
Take this example.
In [123]: import numpy as np
In [124]: np.random.seed(12345)
In [125]: B = 6.0
In [126]: x = np.random.exponential(scale=B, size=5000)
In [127]: from scipy.stats import expon
In [128]: expon.fit(x)
<dozens of warnings clipped>
Out[128]: (0.21874043533906118, 5.7122829778172939)
The fit is achieved by fmin (as far as I know, since disp=0 in the rv_continuous.fit...), but there are a number of warnings emitted. Is there any middle ground to be had in these type of situations via context management perhaps?
Should I file a ticket?
Which numpy scipy versions are you using?
I get know warnings
import numpy as np np.random.seed(12345) B = 6.0 x = np.random.exponential(scale=B, size=5000) from scipy.stats import expon expon.fit(x) array([ 6.43573559e-04, 5.93058867e+00])
from scipy import stats y = [-45, -3, 1, 0, 1, 3] v = stats.norm.pdf(y)/stats.norm.cdf(y)
numpy 1.4.0 scipy 0.8.0b1 Vincent
Numpy
'2.0.0.dev8417'
Scipy
'0.9.0.dev6447'
I don't get any warning with the first example. (numpy 1.4.0) (I cannot run the second example because I have a scipy revision with a broken fit() method)
I don't think wrapping functions/methods to turn off warnings is a good option. (many of them are in inner loops for example for random number generation)
Granted I haven't looked too much into the details of the warnings context manager other than using some toy examples once or twice, but if you could just suppress them for when the solver is called within a function/method then this would do the trick (at least for the ones I have been running into, mostly to do with fitting like this or with maximum likelihood).
Skipper _______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Fri, Jun 11, 2010 at 2:17 PM, Vincent Davis <vincent@vincentdavis.net> wrote:
On Fri, Jun 11, 2010 at 11:34 AM, Skipper Seabold <jsseabold@gmail.com> wrote:
On Fri, Jun 11, 2010 at 1:07 PM, <josef.pktd@gmail.com> wrote:
On Fri, Jun 11, 2010 at 12:45 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
Since the raising of warning behavior has been changed (I believe), I have been running into a lot of warnings in my code when say I do something like
In [120]: from scipy import stats
In [121]: y = [-45, -3, 1, 0, 1, 3]
In [122]: v = stats.norm.pdf(y)/stats.norm.cdf(y) Warning: invalid value encountered in divide
Sometimes, this is useful to know. Sometimes, though, it's very disturbing when it's encountered in some kind of iteration or optimization. I have been using numpy.clip to get around this in my own code, but when it's buried a bit deeper, it's not quite so simple.
Take this example.
In [123]: import numpy as np
In [124]: np.random.seed(12345)
In [125]: B = 6.0
In [126]: x = np.random.exponential(scale=B, size=5000)
In [127]: from scipy.stats import expon
In [128]: expon.fit(x)
<dozens of warnings clipped>
Out[128]: (0.21874043533906118, 5.7122829778172939)
The fit is achieved by fmin (as far as I know, since disp=0 in the rv_continuous.fit...), but there are a number of warnings emitted. Is there any middle ground to be had in these type of situations via context management perhaps?
Should I file a ticket?
Which numpy scipy versions are you using?
I get know warnings
import numpy as np np.random.seed(12345) B = 6.0 x = np.random.exponential(scale=B, size=5000) from scipy.stats import expon expon.fit(x) array([ 6.43573559e-04, 5.93058867e+00])
You also get different values than I do, which shouldn't be the case. I just discovered that my expon.fit(x) just returns the first and second moments of the data (even when I set floc = 0, I still get the second moment), and I am trying to figure out why. It looks like something is amiss. Skipper
On Fri, Jun 11, 2010 at 12:20 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
On Fri, Jun 11, 2010 at 2:17 PM, Vincent Davis <vincent@vincentdavis.net> wrote:
On Fri, Jun 11, 2010 at 11:34 AM, Skipper Seabold <jsseabold@gmail.com> wrote:
On Fri, Jun 11, 2010 at 1:07 PM, <josef.pktd@gmail.com> wrote:
On Fri, Jun 11, 2010 at 12:45 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
Since the raising of warning behavior has been changed (I believe), I have been running into a lot of warnings in my code when say I do something like
In [120]: from scipy import stats
In [121]: y = [-45, -3, 1, 0, 1, 3]
In [122]: v = stats.norm.pdf(y)/stats.norm.cdf(y) Warning: invalid value encountered in divide
Sometimes, this is useful to know. Sometimes, though, it's very disturbing when it's encountered in some kind of iteration or optimization. I have been using numpy.clip to get around this in my own code, but when it's buried a bit deeper, it's not quite so simple.
Take this example.
In [123]: import numpy as np
In [124]: np.random.seed(12345)
In [125]: B = 6.0
In [126]: x = np.random.exponential(scale=B, size=5000)
In [127]: from scipy.stats import expon
In [128]: expon.fit(x)
<dozens of warnings clipped>
Out[128]: (0.21874043533906118, 5.7122829778172939)
The fit is achieved by fmin (as far as I know, since disp=0 in the rv_continuous.fit...), but there are a number of warnings emitted. Is there any middle ground to be had in these type of situations via context management perhaps?
Should I file a ticket?
Which numpy scipy versions are you using?
I get know warnings
import numpy as np np.random.seed(12345) B = 6.0 x = np.random.exponential(scale=B, size=5000) from scipy.stats import expon expon.fit(x) array([ 6.43573559e-04, 5.93058867e+00])
You also get different values than I do, which shouldn't be the case.
I just discovered that my expon.fit(x) just returns the first and second moments of the data (even when I set floc = 0, I still get the second moment), and I am trying to figure out why. It looks like something is amiss.
You also got a tuple back and I got an array, unless you edited. Vincent
Skipper _______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
On Fri, Jun 11, 2010 at 12:34 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
On Fri, Jun 11, 2010 at 1:07 PM, <josef.pktd@gmail.com> wrote:
On Fri, Jun 11, 2010 at 12:45 PM, Skipper Seabold <jsseabold@gmail.com> wrote:
Since the raising of warning behavior has been changed (I believe), I have been running into a lot of warnings in my code when say I do something like
In [120]: from scipy import stats
In [121]: y = [-45, -3, 1, 0, 1, 3]
In [122]: v = stats.norm.pdf(y)/stats.norm.cdf(y) Warning: invalid value encountered in divide
Sometimes, this is useful to know. Sometimes, though, it's very disturbing when it's encountered in some kind of iteration or optimization. I have been using numpy.clip to get around this in my own code, but when it's buried a bit deeper, it's not quite so simple.
Take this example.
In [123]: import numpy as np
In [124]: np.random.seed(12345)
In [125]: B = 6.0
In [126]: x = np.random.exponential(scale=B, size=5000)
In [127]: from scipy.stats import expon
In [128]: expon.fit(x)
<dozens of warnings clipped>
Out[128]: (0.21874043533906118, 5.7122829778172939)
The fit is achieved by fmin (as far as I know, since disp=0 in the rv_continuous.fit...), but there are a number of warnings emitted. Is there any middle ground to be had in these type of situations via context management perhaps?
Should I file a ticket?
Which numpy scipy versions are you using?
Numpy
'2.0.0.dev8417'
Scipy
'0.9.0.dev6447'
I don't get any warning with the first example. (numpy 1.4.0) (I cannot run the second example because I have a scipy revision with a broken fit() method)
I don't think wrapping functions/methods to turn off warnings is a good option. (many of them are in inner loops for example for random number generation)
Granted I haven't looked too much into the details of the warnings context manager other than using some toy examples once or twice, but if you could just suppress them for when the solver is called within a function/method then this would do the trick (at least for the ones I have been running into, mostly to do with fitting like this or with maximum likelihood).
Skipper
Replying to myself here. For noiseless tests (ie., no floating point warnings) just do import numpy as np np.seterr(all='ignore') Skipper
participants (3)
-
josef.pktd＠gmail.com -
Skipper Seabold -
Vincent Davis