[scikit-learn] Why ridge regression can solve multicollinearity?

[josef.pktd at gmail.com]

On Wed, Jan 8, 2020 at 9:43 PM <josef.pktd at gmail.com> wrote:

>
>
> On Wed, Jan 8, 2020 at 9:38 PM lampahome <pahome.chen at mirlab.org> wrote:
>
>>
>>
>> Stuart Reynolds <stuart at stuartreynolds.net> 於 2020年1月9日 週四 上午10:33寫道：
>>
>>> Correlated features typically have the property that they are tending to
>>> be similarly predictive of the outcome.
>>>
>>> L1 and L2 are both a preference for low coefficients.
>>> If a coefficient can be reduced yet another coefficient maintains
>>> similar loss, the these regularization methods prefer this solution.
>>> If you use L1 or L2, you should mean and variance normalize your
>>> features.
>>>
>>>
>> You mean LASSO and RIDGE both solve multilinearity?
>>
>
> LASSO has the reputation not to be good when there is multicollinearity,
> that's why elastic net L1 + L2 was introduced, AFAIK
>
> With multicollinearity the length of the parameter vector, beta' beta, is
> too large and L2, Ridge shrinks it.
>

e.g.
Marquardt, Donald W., and Ronald D. Snee. "Ridge regression in practice." *The
American Statistician* 29, no. 1 (1975): 3-20.

I just went through it last week because of a argument about variance
inflation factor in Ridge




>
> Josef
>
>
>
>>
>> _______________________________________________
>> scikit-learn mailing list
>> scikit-learn at python.org
>> https://mail.python.org/mailman/listinfo/scikit-learn
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20200108/92f61a02/attachment-0001.html>