On Thu, Mar 27, 2008 at 12:57 PM, massimo sandal <massimo.sandal@unibo.it> wrote:
Xavier Gnata ha scritto:
> Well think about it one second : A  fit with a 12 order polynomial will
> always have a smaller least mean square error the a fit using a <12
> order polynomial. Your must figure out we before using a polyfit.

Well, this is reasonable but it is not what I see happening. Often the
algorithm takes the highest-degree polynomial, but not always.

Hum we have a problem here:
Polyfit is a linear fit. It can (should?) be implemented using using the pseudo-inverse paradigm and it "pseudo-inverse provide you with the solution that minimize the chi2. As a result, a 12 ordrer fit cannot be worst than a 5 order. It should at least end up with the same chi2.

One trivial exemple :
In [9]: polyfit(arange(1000),2*arange(1000)+1,30)
C:\Python25\lib\site-packages\numpy\lib\polynomial.py:306: RankWarning:
Polyfit
may be poorly conditioned
warnings.warn(msg, RankWarning)
Out[9]:
array([ 4.25295067e-93, -1.90319088e-89, 2.47204987e-86,
3.64986075e-84, -2.04058052e-80, -1.02156478e-77,
1.38333948e-74, 1.84191780e-71, -1.04858370e-69,
-1.93210697e-65, -1.30453334e-62, 1.07062138e-59,
2.11173521e-56, 2.00842417e-54, -2.16681833e-50,
-1.21709485e-47, 1.99280602e-44, 1.67639356e-41,
-2.35550438e-38, -1.10826788e-35, 3.53529389e-32,
-2.99768374e-29, 1.44456488e-26, -4.49042306e-24,
9.31940018e-22, -1.28887179e-19, 1.15776067e-17,
-6.43505943e-16, 2.04266284e-14, 2.00000000e+00,
1.00000000e+00])
Of course, we also have numerical errors but if they cannot be neglected, then to have found a problem which is really ill-conditioned (hudge range of values ? values very close to inf or 0 ?)...or there is a pb in poyfit but I cannot see this bug.

Any comments?

Xavier

 

> Moreover, "smaller least mean square error" is not the driver in our
> case because high order polynomials show (it is very common) large
> oscillations in between the data points. That is not what you want.

Yes, I know. However it seems to behave extremly well in this case -I
tested on dozens of data sets and it flats them all practically
perfectly for my needs. That's why I asked -should I worry *even if it
works*?

> I'm happy to see your question because the worth behavior is to use a
> tool blindly.

You probably meant "worst", however yes, I agree :)

m.

--
Massimo Sandal
University of Bologna
Department of Biochemistry "G.Moruzzi"

snail mail:
Via Irnerio 48, 40126 Bologna, Italy

email:
massimo.sandal@unibo.it

tel: +39-051-2094388
fax: +39-051-2094387

_______________________________________________
SciPy-user mailing list
SciPy-user@scipy.org
http://projects.scipy.org/mailman/listinfo/scipy-user