<div dir="ltr"><div class="gmail_default" style="color:rgb(0,0,0)"><font size="4"></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font size="4">Thanks again for the clarifications Sebastian!</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font size="4"><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font size="4">Keras has a Scikit-learn API with the KeraRegressor which implements the Scikit-Learn MLPRegressor interface:</font></div><div class="gmail_default" style="color:rgb(0,0,0)"><font size="4"><br></font></div><div class="gmail_default"><font size="4"><font color="#242729" face="Arial, Helvetica Neue, Helvetica, sans-serif"><a href="https://keras.io/scikit-learn-api/">https://keras.io/scikit-learn-api/</a></font><br></font></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="color:rgb(36,39,41);font-family:Arial,"Helvetica Neue",Helvetica,sans-serif"><font size="4"><br></font></span></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="color:rgb(36,39,41);font-family:Arial,"Helvetica Neue",Helvetica,sans-serif"><font size="4">Is it possible to change the loss function in KerasRegressor? I don't have time right now to experiment with hyperparameters of new ANN architectures. I am in urgent need to reproduce in Keras the results obtained with MLPRegressor and the set of hyperparameters that I have optimized for my problem and later change the loss function.</font></span></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="color:rgb(36,39,41);font-family:Arial,"Helvetica Neue",Helvetica,sans-serif"><font size="4"><br></font></span></div><div class="gmail_default" style="color:rgb(0,0,0)"><span style="color:rgb(36,39,41);font-family:Arial,"Helvetica Neue",Helvetica,sans-serif"><font size="4"><br></font></span></div><div class="gmail_extra"><font size="4"><br></font><div class="gmail_quote"><font size="4">On 13 September 2017 at 18:14, Sebastian Raschka <span dir="ltr"><<a href="mailto:se.raschka@gmail.com" target="_blank">se.raschka@gmail.com</a>></span> wrote:<br></font><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><font size="4"><span class="">> What about the SVR? Is it possible to change the loss function there?<br>
<br>
</span>Here you would have the same problem; SVR is a constrained optimization problem and you would have to change the calculation of the loss gradient then. Since SVR is a "1-layer" neural net, if you change the cost function to something else, it's not really a SVR anymore.<br>
<span class=""><br>
<br>
> Could you please clarify what the "x" and "x'" parameters in the default Kernel functions mean? Is "x" a NxM array, where N is the number of observations and M the number of features?<br>
<br>
</span>Both x and x' should be denoting training examples. The kernel matrix is symmetric (N x N).<br>
<br>
<br>
<br>
Best,<br>
Sebastian<br>
</font><div class="HOEnZb"><div class="h5"><font size="4"><br>
> On Sep 13, 2017, at 5:25 AM, Thomas Evangelidis <<a href="mailto:tevang3@gmail.com">tevang3@gmail.com</a>> wrote:<br>
><br>
> Thanks Sebastian. Exploring Tensorflow capabilities was in my TODO list, but now it's in my immediate plans.<br>
> What about the SVR? Is it possible to change the loss function there? Could you please clarify what the "x" and "x'" parameters in the default Kernel functions mean? Is "x" a NxM array, where N is the number of observations and M the number of features?<br>
><br>
> <a href="http://scikit-learn.org/stable/modules/svm.html#kernel-functions" rel="noreferrer" target="_blank">http://scikit-learn.org/<wbr>stable/modules/svm.html#<wbr>kernel-functions</a><br>
><br>
><br>
><br>
> On 12 September 2017 at 00:37, Sebastian Raschka <<a href="mailto:se.raschka@gmail.com">se.raschka@gmail.com</a>> wrote:<br>
> Hi Thomas,<br>
><br>
> > For the MLPRegressor case so far my conclusion was that it is not possible unless you modify the source code.<br>
><br>
> Also, I suspect that this would be non-trivial. I haven't looked to closely at how the MLPClassifier/MLPRegressor are implemented but since you perform the weight updates based on the gradient of the cost function wrt the weights, the modification would be non-trivial if the partial derivatives are not computed based on some autodiff implementation -- you would have to edit all the partial d's along the backpropagation up to the first hidden layer. While I think that scikit-learn is by far the best library out there for machine learning, I think if you want an easy solution, you probably won't get around TensorFlow or PyTorch or equivalent, here, for your specific MLP problem unless you want to make your life extra hard :P (seriously, you can pick up any of the two in about an hour and have your MLPRegressor up and running so that you can then experiment with your cost function).<br>
><br>
> Best,<br>
> Sebastian<br>
><br>
> > On Sep 11, 2017, at 6:13 PM, Thomas Evangelidis <<a href="mailto:tevang3@gmail.com">tevang3@gmail.com</a>> wrote:<br>
> ><br>
> > Greetings,<br>
> ><br>
> > I know this is a recurrent question, but I would like to use my own loss function either in a MLPRegressor or in an SVR. For the MLPRegressor case so far my conclusion was that it is not possible unless you modify the source code. On the other hand, for the SVR I was looking at setting custom kernel functions. But I am not sure if this is the same thing. Could someone please clarify this to me? Finally, I read about the "scoring" parameter is cross-validation, but this is just to select a Regressor that has been trained already with the default loss function, so it would be harder to find one that minimizes my own loss function.<br>
> ><br>
> > For the record, my loss function is the centered root mean square error.<br>
> ><br>
> > Thanks in advance for any advice.<br>
> ><br>
> ><br>
> ><br>
> > --<br>
> > ==============================<wbr>==============================<wbr>==========<br>
> > Dr Thomas Evangelidis<br>
> > Post-doctoral Researcher<br>
> > CEITEC - Central European Institute of Technology<br>
> > Masaryk University<br>
> > Kamenice 5/A35/2S049,<br>
> > 62500 Brno, Czech Republic<br>
> ><br>
> > email: <a href="mailto:tevang@pharm.uoa.gr">tevang@pharm.uoa.gr</a><br>
> > <a href="mailto:tevang3@gmail.com">tevang3@gmail.com</a><br>
> ><br>
> > website: <a href="https://sites.google.com/site/thomasevangelidishomepage/" rel="noreferrer" target="_blank">https://sites.google.com/site/<wbr>thomasevangelidishomepage/</a><br>
> ><br>
> ><br>
> > ______________________________<wbr>_________________<br>
> > scikit-learn mailing list<br>
> > <a href="mailto:scikit-learn@python.org">scikit-learn@python.org</a><br>
> > <a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/scikit-learn</a><br>
><br>
> ______________________________<wbr>_________________<br>
> scikit-learn mailing list<br>
> <a href="mailto:scikit-learn@python.org">scikit-learn@python.org</a><br>
> <a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/scikit-learn</a><br>
><br>
><br>
><br>
> --<br>
> ==============================<wbr>==============================<wbr>==========<br>
> Dr Thomas Evangelidis<br>
> Post-doctoral Researcher<br>
> CEITEC - Central European Institute of Technology<br>
> Masaryk University<br>
> Kamenice 5/A35/2S049,<br>
> 62500 Brno, Czech Republic<br>
><br>
> email: <a href="mailto:tevang@pharm.uoa.gr">tevang@pharm.uoa.gr</a><br>
> <a href="mailto:tevang3@gmail.com">tevang3@gmail.com</a><br>
><br>
> website: <a href="https://sites.google.com/site/thomasevangelidishomepage/" rel="noreferrer" target="_blank">https://sites.google.com/site/<wbr>thomasevangelidishomepage/</a><br>
><br>
><br>
> ______________________________<wbr>_________________<br>
> scikit-learn mailing list<br>
> <a href="mailto:scikit-learn@python.org">scikit-learn@python.org</a><br>
> <a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/scikit-learn</a><br>
<br>
______________________________<wbr>_________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/<wbr>mailman/listinfo/scikit-learn</a><br>
</font></div></div></blockquote></div><font size="4"><br><br clear="all"></font><div><font size="4"><br></font></div><font size="4">-- <br></font><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div>
<p style="margin-bottom:0cm" align="LEFT"><span style="color:rgb(0,0,0)"><span style="font-family:arial,helvetica,sans-serif"><font size="4">======================================================================</font></span></span></p>
<p style="margin-bottom:0cm" align="LEFT"><span style="color:rgb(0,0,0)"><span style="font-family:arial,helvetica,sans-serif"><font size="4">Dr Thomas Evangelidis</font></span></span></p>
<p style="margin-bottom:0cm" align="LEFT"><span style="color:rgb(0,0,0)"><span style="font-family:arial,helvetica,sans-serif"><font size="4">Post-doctoral Researcher<br></font></span></span></p><span style="color:rgb(0,0,0)"><span style="font-family:arial,helvetica,sans-serif"><font size="4">CEITEC - Central European Institute of Technology<br>Masaryk University<br>Kamenice 5/A35/2S049, <br>62500 Brno, Czech Republic <br></font></span></span></div><div dir="ltr"><span style="color:rgb(0,0,0)"><font size="4"><br></font></span><p style="margin-bottom:0cm" align="LEFT"><span style="color:rgb(0,0,0)"><span style="font-family:arial,helvetica,sans-serif"><font size="4">email: <a href="mailto:tevang@pharm.uoa.gr" target="_blank">tevang@pharm.uoa.gr</a></font></span></span></p>
<p style="margin-bottom:0cm" align="LEFT"><span style="color:rgb(0,0,0)"><span style="font-family:arial,helvetica,sans-serif"><font size="4"> <a href="mailto:tevang3@gmail.com" target="_blank">tevang3@gmail.com</a></font></span></span></p>
<p style="margin-bottom:0cm" align="LEFT"><span style="color:rgb(0,0,0)"><span style="font-family:arial,helvetica,sans-serif"><font size="4"><br>website:
<a href="https://sites.google.com/site/thomasevangelidishomepage/" target="_blank">https://sites.google.com/site/thomasevangelidishomepage/</a></font></span></span></p><span style="color:rgb(0,0,0)"><br></span><p style="margin-bottom:0cm" align="LEFT">
</p>
</div></div></div></div></div></div></div></div>
</div></div>