[scikit-learn] Ordinary Least Square Regression Under-determined system.

[josef.pktd at gmail.com]

scipy.linalg.leastq uses an SVD solver and drops singular components,
where singular depends on the condition number threshold.

So it's equivalent to PCR with a tiny threshold for dropping
components (rcond < 1e-15, if it's similar to numpy). SVD/rcond is on
original, not on standardized variables.

The PCA in PCR is not supervised in contrast to PLS (at least in
general, I have no idea about scikit-learn versions)

Josef

On Tue, Apr 25, 2017 at 12:40 PM, Andreas Mueller <t3kcit at gmail.com> wrote:
> PLS is not the same as PCR, right? Why did you expect them to perform the
> same?
> LinearRegression is just calling scipy.linalg.lstsq
> https://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.lstsq.html
> as you can see here:
> https://github.com/scikit-learn/scikit-learn/blob/14031f6/sklearn/linear_model/base.py#L539
> I expect that produces the minimum norm solution though I'm not familiar
> with the exact solver.
>
>
>
>
> On 04/16/2017 01:15 PM, Joshua Mannheimer wrote:
>
> Hi all,
>
> So I am trying to write a Principle Components Regression implementation in
> Python to match the PLS package in R. I am getting better results in R so I
> am trying to figure out where the discrepancy was. The data I am using is
> way undetermined where n_features ~ 50,000 and n_samples ~ 500 thus why PCR
> is necessary. Just to see what would happen I used sklearn.LinearRegression
> on the original 500 X 50000 dataset. I expected I would get an error message
> stating the the system was not solvable but it worked and I got an answer
> that was at least on par with the PCR solution. So I am wondering how it is
> possibly solving this system if anybody knows. Thanks
>
> --
> Joshua D. Mannheimer M.E.
> Biomedical Engineering Ph.d Student
> Flint Animal Cancer Research Center
> Office: A 259 CSU Veterinary Campus
> Colorado State University
> (970)-389-3951
> jmannhei at rams.colostate.edu
>
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
>
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>