On Wed, Mar 26, 2008 at 5:55 PM, massimo sandal <massimo.sandal@unibo.it> wrote:
Hi,

I am writing an algorithm that determines the best-fitting polynomial to
a data set[1]. The algorithm tries to fit polynomials between 1st and
12th degree, and chooses the polynomial with the least mean square error.

It works really well now, however when it occurs to prefer high-grade
polynomials (say, 8th-9th grade and more), I often see this warning:

/usr/lib/python2.5/site-packages/numpy/lib/polynomial.py:305:
RankWarning: Polyfit may be poorly conditioned
  warnings.warn(msg, RankWarning)

I have poor knowledge on polynomial fitting, but if I remember
correctly, it means that the fit is very sensitive to tiny shifts in the
data values (unstable)? Should I worry anyway or, as long as it gives me
sensible results, go along?

Thanks,
Massimo

[1]this is needed to "flatten" the data set from systematic, irregular
low-frequency interferences.
--
Massimo Sandal
University of Bologna
Department of Biochemistry "G.Moruzzi"

snail mail:
Via Irnerio 48, 40126 Bologna, Italy

email:
massimo.sandal@unibo.it

tel: +39-051-2094388
fax: +39-051-2094387

Hi,

Well think about it one second : A  fit with a 12 order polynomial will always have a smaller least mean square error the a fit using a <12 order polynomial. Your must figure out we before using a polyfit.

Moreover, "smaller least mean square error" is not the driver in our case because high order polynomials show (it is very common) large oscillations in between the data points. That is not what you want.

I'm happy to see your question because the worth behavior is to use a tool blindly.

Xavier

Xavier

Xavier