[scikit-learn] Why ridge regression can solve multicollinearity?

josef.pktd at gmail.com josef.pktd at gmail.com
Wed Jan 8 21:43:54 EST 2020

On Wed, Jan 8, 2020 at 9:38 PM lampahome <pahome.chen at mirlab.org> wrote:

> Stuart Reynolds <stuart at stuartreynolds.net> 於 2020年1月9日 週四 上午10:33寫道:
>> Correlated features typically have the property that they are tending to
>> be similarly predictive of the outcome.
>> L1 and L2 are both a preference for low coefficients.
>> If a coefficient can be reduced yet another coefficient maintains similar
>> loss, the these regularization methods prefer this solution.
>> If you use L1 or L2, you should mean and variance normalize your features.
> You mean LASSO and RIDGE both solve multilinearity?

LASSO has the reputation not to be good when there is multicollinearity,
that's why elastic net L1 + L2 was introduced, AFAIK

With multicollinearity the length of the parameter vector, beta' beta, is
too large and L2, Ridge shrinks it.


> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20200108/b959d061/attachment.html>

More information about the scikit-learn mailing list