<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
<p>I use</p>
<p>sum((cross_val_predict(model, X, y) - y)**2) / len(y) (*)<br>
</p>
<p>to evaluate the performance of a model. This conforms with
Murphy: Machine Learning, section 6.5.3, and Hastie et al: The
Elements of Statistical Learning, eq. 7.48. However, according to
the documentation of cross_val_predict, "it is not appropriate to
pass these predictions into an evaluation metric". While it is
obvious that cross_val_predict is different from cross_val_score,
I don't see what should be wrong with (*).</p>
<p>Also, the explanation that "<a class="reference internal"
href="https://scikit-learn.org/stable/modules/generated/sklearn.model_selection.cross_val_predict.html#sklearn.model_selection.cross_val_predict"
title="sklearn.model_selection.cross_val_predict" style="color:
rgb(40, 120, 162); text-decoration: none; overflow-wrap:
break-word; font-family: Helvetica, Arial, sans-serif;
font-size: 14.4px; font-style: normal; font-variant-ligatures:
normal; font-variant-caps: normal; font-weight: 400;
letter-spacing: normal; orphans: 2; text-align: start;
text-indent: 0px; text-transform: none; white-space: normal;
widows: 2; word-spacing: 0px; -webkit-text-stroke-width: 0px;
background-color: rgb(243, 229, 229);"><code class="xref py
py-func docutils literal" style="padding: 2px 4px;
font-family: monospace; font-size: 1.1em; color: inherit;
border-radius: 3px; background-color: transparent; border:
none; white-space: nowrap; font-weight: bold;"><span
class="pre" style="hyphens: none;">cross_val_predict</span></code></a><span
style="color: rgb(185, 74, 72); font-family: Helvetica, Arial,
sans-serif; font-size: 14.4px; font-style: normal;
font-variant-ligatures: normal; font-variant-caps: normal;
font-weight: 400; letter-spacing: normal; orphans: 2;
text-align: start; text-indent: 0px; text-transform: none;
white-space: normal; widows: 2; word-spacing: 0px;
-webkit-text-stroke-width: 0px; background-color: rgb(243, 229,
229); text-decoration-style: initial; text-decoration-color:
initial; display: inline !important; float: none;"><span> </span>simply
returns the labels (or probabilities)</span>" is unclear, if not
wrong. As I understand it, this function returns estimates and no
labels or probabilities.</p>
<p>Regards, Boris<br>
</p>
</body>
</html>