<div dir="auto">Indeed , this is basically what I told you (you do not be need to copy textbook stuff: I taught probas/stats) : these are mostly problems for *inference*. </div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, 13 Aug 2021, 12:03 Samir K Mahajan, <<a href="mailto:samirkmahajan1972@gmail.com">samirkmahajan1972@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><h3 style="overflow:hidden;white-space:nowrap;font-size:0.75rem;margin:inherit;text-overflow:ellipsis;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;letter-spacing:0.3px;color:rgb(95,99,104);line-height:20px"><br></h3><div><h3 style="text-align:justify;line-height:15pt;margin:2pt 0cm 0cm;break-after:avoid;font-size:12pt;font-family:"Calibri Light",sans-serif;color:rgb(31,55,99);font-weight:normal"><span><span style="font-size:11pt;font-family:"Times New Roman",serif;color:rgb(32,33,36);letter-spacing:0.15pt">Dear Christophe Pallier<b>,</b></span></span><span style="font-size:11pt;font-family:"Times New Roman",serif;color:rgb(95,99,104);letter-spacing:0.25pt"></span></h3>
<p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 8pt;line-height:107%;font-size:11pt;font-family:Calibri,sans-serif"><span><span name="Christophe Pallier" style="font-size:0.875rem"><span style="font-family:"Times New Roman",serif;color:rgb(32,33,36);letter-spacing:0.15pt"><span role="gridcell" style="outline:none">When we are doing prediction, we are relying on the
values of the coefficients of the model created. We are feeding test data on the model for prediction. We may be nterested to see if the OLS estimators(coefficients) are BLUE or
not. In the presence of autocorrelation (normally noticed in time series data), residuals are not independent, and as
such the OLS estimators are not BLUE in the sense that they don't have
minimum variance, and thus no more efficient estimators. Statistical tests (t, F and </span></span></span></span><i><span style="font-family:"Times New Roman",serif">χ</span></i><span style="font-family:"Times New Roman",serif">2) may not be
valid. We may reject the model to make predictions in such a situation. . We have to rely upon other improved models. There may be issues relating to multicollinearity (in case of multivariable regression model) and heteroscedasticity (mostly seen in cross-section data) too in a model. Can we discard these tools while predicting a model? </span></p><p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 8pt;line-height:107%;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Times New Roman",serif">Regards,</span></p><p class="MsoNormal" style="text-align:justify;margin:0cm 0cm 8pt;line-height:107%;font-size:11pt;font-family:Calibri,sans-serif"><span style="font-family:"Times New Roman",serif">Samir K Mahajan </span></p></div><div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 13, 2021 at 1:07 PM Christophe Pallier <<a href="mailto:christophe@pallier.org" target="_blank" rel="noreferrer">christophe@pallier.org</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto">Actually, multicollinearity and autocorrelation are problems for *inference* more than for *prediction*. For example, if there is autocorrelation, the residuals are not independent, and the degrees of freedom are wrong for the tests in an OLS model (but you can use, e.g., an AR1 model). </div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Thu, 12 Aug 2021, 22:32 Samir K Mahajan, <<a href="mailto:samirkmahajan1972@gmail.com" rel="noreferrer noreferrer" target="_blank">samirkmahajan1972@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">A note please (to Sebastian Raschka, mrschots). <br><br><br><table cellpadding="0" role="presentation" style="border-collapse:collapse;margin-top:0px;width:auto;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:0.875rem;letter-spacing:0.2px;display:block"></table> The OLS model that I used ( where the test score gave me a negative value) was not a good fit.<b> </b> Initial findings showed that t<b>he regression coefficients and the model as a whole were significant, </b>yet ,
finally , it failed in two econometrics tests such as VIF (used for detecting multicollinearity ) and Durbin-Watson test ( used for detecting auto-correlation). <b>Presence of multicollinearity and autocorrelation problems </b> in the model make it unsuitable for prediction. <div>Regards, <div><br></div><div>Samir K Mahajan. </div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 13, 2021 at 1:41 AM Samir K Mahajan <<a href="mailto:samirkmahajan1972@gmail.com" rel="noreferrer noreferrer noreferrer" target="_blank">samirkmahajan1972@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Thanks to all of you for your kind response. Indeed, it is a great learning experience. Yes, econometrics books too create models for prediction, and programming really makes things better in a complex world. My understanding is that machine learning does depend on econometrics too. <div><div><br></div><div>My Regards, </div><div><br></div><div>Samir K Mahajan </div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Fri, Aug 13, 2021 at 1:21 AM Sebastian Raschka <<a href="mailto:mail@sebastianraschka.com" rel="noreferrer noreferrer noreferrer" target="_blank">mail@sebastianraschka.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div>
<div name="messageBodySection">
<div dir="auto">The R2 function in scikit-learn works fine. A negative means that the regression model fits the data worse than a horizontal line representing the sample mean. E.g. you usually get that if you are overfitting the training set a lot and then apply that model to the test set. The econometrics book probably didn't cover applying a model to an independent data or test set, hence the [0, 1] suggestion.<br>
<br>
Cheers,<br>
Sebastian<br>
<br>
<br></div>
</div>
<div name="messageReplySection">On Aug 12, 2021, 2:20 PM -0500, Samir K Mahajan <<a href="mailto:samirkmahajan1972@gmail.com" rel="noreferrer noreferrer noreferrer" target="_blank">samirkmahajan1972@gmail.com</a>>, wrote:<br>
<blockquote type="cite" style="border-left:thin solid grey;margin:5px;padding-left:10px">
<div dir="ltr">
<div><br></div>
Dear <span style="color:rgb(32,33,36);font-size:0.875rem;letter-spacing:0.2px;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;white-space:nowrap">Christophe Pallier, </span> Reshama Saikh and Tromek Drabas,
<h3 style="overflow:hidden;white-space:nowrap;font-size:0.75rem;font-weight:inherit;margin:inherit;text-overflow:ellipsis;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;letter-spacing:0.3px;color:rgb(95,99,104);line-height:20px"><br></h3>
<div><font color="#202124" face="Roboto, RobotoDraft, Helvetica, Arial, sans-serif"><span style="font-size:14px;letter-spacing:0.2px;white-space:nowrap">Thank you for your kind response. </span></font> Fair enough. I go with you R2 is not a square. However, if you open any book of econometrics, it says R2 is a ratio that lies between 0 and 1. <b>This is the constraint.</b> It measures the proportion or percentage of the total variation in response variable (Y) explained by the regressors (Xs) in the model . Remaining proportion of variation in Y, if any, is explained by the residual term(u) Now, sklearn.matrics. metrics.r2_score gives me a negative value lying on a linear scale (<span style="color:rgb(0,0,0);font-size:14px;white-space:pre-wrap">-5.763335245921777). This negative value breaks the <b>constraint.</b> I just want to highlight that. I think it needs to be corrected.</span> <span style="color:rgb(0,0,0);font-size:14px;white-space:pre-wrap">Rest is up to you .</span></div>
<div><span style="color:rgb(0,0,0);font-size:14px;white-space:pre-wrap"><br></span></div>
<div>I find that Reshama Saikh is hurt by my email. I am really sorry for that. Please note I never undermine your capabilities and initiatives. You are great people doing great jobs. I realise that I should have been more sensible. </div>
<div><br></div>
<div>My regards to all of you.</div>
<div><br></div>
<div>Samir K Mahajan <br>
<table cellpadding="0" style="border-collapse:collapse;margin-top:0px;width:auto;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:14px;letter-spacing:0.2px;display:block">
<tbody style="display:block">
<tr style="height:auto;display:flex">
<td style="white-space:nowrap;padding:0px;vertical-align:top;width:844.55px;line-height:20px;display:block;max-height:20px"><br></td>
</tr>
</tbody>
</table>
<br>
<table cellpadding="0" style="border-collapse:collapse;margin-top:0px;width:auto;font-family:Roboto,RobotoDraft,Helvetica,Arial,sans-serif;font-size:14px;letter-spacing:0.2px;display:block">
<tbody style="display:block"></tbody>
</table>
</div>
<div><span style="color:rgb(0,0,0);font-size:14px;white-space:pre-wrap"><br></span></div>
<div><span style="color:rgb(0,0,0);font-size:14px;white-space:pre-wrap"><br></span></div>
<div><font color="#000000"><span style="font-size:14px;white-space:pre-wrap"><br></span></font>
<div><span style="color:rgb(0,0,0);font-size:14px;white-space:pre-wrap"><br></span></div>
<div><span style="color:rgb(0,0,0);font-size:14px;white-space:pre-wrap"><br></span></div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Thu, Aug 12, 2021 at 12:02 PM Christophe Pallier <<a href="mailto:christophe@pallier.org" rel="noreferrer noreferrer noreferrer" target="_blank">christophe@pallier.org</a>> wrote:<br></div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="auto">Simple: despite its name R2 is not a square. Look up its definition.</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Wed, 11 Aug 2021, 21:17 Samir K Mahajan, <<a href="mailto:samirkmahajan1972@gmail.com" rel="noreferrer noreferrer noreferrer" target="_blank">samirkmahajan1972@gmail.com</a>> wrote:<br></div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div>Dear All,</div>
<div>I am amazed to find negative values of sklearn.metrics.r2_score and sklearn.metrics.explained_variance_score in a model ( cross validation of OLS regression model) </div>
<div>However, what amuses me more is seeing you justifying negative 'sklearn.metrics.r2_score ' in your documentation. This does not make sense to me . Please justify to me how squared values are negative. </div>
<div><br></div>
<div>Regards,</div>
<div>Samir K Mahajan. </div>
<div></div>
<div><br></div>
</div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer noreferrer noreferrer noreferrer noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br></blockquote>
</div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" rel="noreferrer noreferrer noreferrer" target="_blank">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br></blockquote>
</div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" rel="noreferrer noreferrer noreferrer" target="_blank">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer noreferrer noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br></blockquote>
</div>
</div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" rel="noreferrer noreferrer noreferrer" target="_blank">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote></div>
</blockquote></div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" rel="noreferrer noreferrer noreferrer" target="_blank">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer noreferrer noreferrer noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote></div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank" rel="noreferrer">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote></div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank" rel="noreferrer">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote></div>