rcond in polyfit

[A. M. Archibald]

On 14/10/06, Charles R Harris <charlesr.harris at gmail.com> wrote:
>
>
> On 10/13/06, A. M. Archibald <peridot.faceted at gmail.com> wrote:
> > On 13/10/06, Tim Hochberg <tim.hochberg at ieee.org> wrote:
> > > Charles R Harris wrote:
>
> <snip>
>
> > > On the other hand if error handling is set to 'raise', then a
> > > FloatingPointError is issued. Is a FloatingPointWarning in order to
> > > mirror the FloatingPointError? And if so, would it be appropriate to use
> > > for condition number?
> >
> > I submitted a patchto use warnings for several functions in scipy a
> > while ago, and the approach I took was to create a ScipyWarning, from
> > which more specific warnings were derived (IntegrationWarning, for
> > example). That was perhaps a bit short-sighted.
> >
> > I'd suggest a FloatingPointWarning as a base class, with
> > IllConditionedMatrix as a subclass (it should include the condition
> > number, but probably not the matrix itself unless it's small, as
> > debugging information).
> >
>
> Let's pin this down a bit. Numpy seems to need its own warning classes so we
> can control the printing of the warnings. For the polyfit function I think
> it should *always* warn by default because it is likely to be used
> interactively and one can fool around with the degree of the fit. For
> instance, I am now scaling the the input x vector so norm_inf(x) == 1 and
> this seems to work pretty well for lots of stuff with rcond=-1 (about 1e-16)
> and a warning on rank reduction. As long as the rank stays the same things
> seem to work ok, up to fits of degree 21 on the test data that started this
> conversation.

Numerical Recipes (http://www.nrbook.com/a/bookcpdf/c15-4.pdf )
recommend setting rcond to the number of data points times machine
epsilon (which of course is different for single/double). We should
definitely warn the user if any singular value is below s[0]*rcond (as
that means that there is effectively a useless basis function, up to
roundoff).

I'm not sure how to control the default warnings setting ("once" vs.
"always"); it's definitely not possible using the standard API to save
the warnings state and restore it later. One might be able to push
such a change into the warnings module by including it in a
ContextManager.

ipython should probably reset all the "show once" warnings every time
it shows an interactive prompt. I suppose more accurately, it should
do that only for warnings the user hasn't given instructions about.
That way you'll get a warning about bad polynomial fits every time you
run a command that contains one, but if your function runs thousands
of fits you don't drown in warnings.

> BTW, how does one turn warnings back on? If I do a
>
> >>> warnings.simplefilter('always', mywarn)
>
> things work fine. Following this by
>
> >>> warnings.simplefilter('once', mywarn)
>
> does what is supposed to do. Once again issuing
>
> >>> warnings.simplefilter('always', mywarn)
>
> fails to have any effect. Resetwarnings doesn't help. Hmmm...

I don't get the impression that the warnings module is much tested; I
had similar headaches.

A. M. Archibald

-------------------------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642