[SciPy-User] adding linear fitting routine

[David Pine]

On Dec 4, 2013, at 3:24 PM, josef.pktd at gmail.com wrote:

> 
> linfit looks like an enhanced version of linregress, which also has
> only one regressor, but doesn't have weights
> http://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.linregress.html

The problem with this is that the statistical tests that linregress uses--the r-value, p-value, & stderr-- are not really compatible with the weighted chi-squared fitting that linfit does.  The  r-value, p-value, & stderr are statistical tests that are used mostly in the social sciences (see http://en.wikipedia.org/wiki/Coefficient_of_determination).  Looking at linregress, it's clear that it was written with that community in mind.

By contrast, linfit (and curve_fit) use the chi-squared measure of goodness of fit, which is explicitly made to be used with weighted data.  In my opinion, trying to satisfy the needs of both communities with one function will result in inefficient code and confusion in both user communities.  linfit naturally goes with the curve_fit and polyfit functions, and is implemented consistent with those fitting routines.  linregress is really a different animal, with statistical tests normally used with unweighted data, and I suspect that the community that uses it will be put off by the "improvements" made by linfit.

> 
> relsigma is similar to the new `absolute_sigma` in curve_fit
> https://github.com/scipy/scipy/pull/3098

That's right.  linfit implements essentially the same functionality that is being implemented in curve_fit

> 
> 
> I think linregress could be rewritten to include these improvements.
> 
> Otherwise I keep out of any fitting debates, because I think `odr` is
> better for handling measurement errors in the x variables, and
> statsmodels is better for everything else (mainly linear only so far)
> and `lmfit` for nonlinear LS.
> There might be a case for stripped down convenience functions or
> special case functions.
> 
> Josef

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20131204/2e815b12/attachment.html>