[scikit-learn] custom loss function

[Sebastian Raschka]

> What about the SVR? Is it possible to change the loss function there?

Here you would have the same problem; SVR is a constrained optimization problem and you would have to change the calculation of the loss gradient then. Since SVR is a "1-layer" neural net, if you change the cost function to something else, it's not really a SVR anymore.


> Could you please clarify what the "x" and "x'" parameters in the default Kernel functions mean? Is "x" a NxM array, where N is the number of observations and M the number of features?

Both x and x' should be denoting training examples. The kernel matrix is symmetric (N x N).



Best,
Sebastian

> On Sep 13, 2017, at 5:25 AM, Thomas Evangelidis <tevang3 at gmail.com> wrote:
> 
> Thanks Sebastian. Exploring Tensorflow capabilities was in my TODO list, but now it's in my immediate plans.
> What about the SVR? Is it possible to change the loss function there? Could you please clarify what the "x" and "x'" parameters in the default Kernel functions mean? Is "x" a NxM array, where N is the number of observations and M the number of features?
> 
> http://scikit-learn.org/stable/modules/svm.html#kernel-functions
> 
> 
> 
> On 12 September 2017 at 00:37, Sebastian Raschka <se.raschka at gmail.com> wrote:
> Hi Thomas,
> 
> > For the MLPRegressor case so far my conclusion was that it is not possible unless you modify the source code.
> 
> Also, I suspect that this would be non-trivial. I haven't looked to closely at how the MLPClassifier/MLPRegressor are implemented but since you perform the weight updates based on the gradient of the cost function wrt the weights, the modification would be non-trivial if the partial derivatives are not computed based on some autodiff implementation -- you would have to edit all the partial d's along the backpropagation up to the first hidden layer. While I think that scikit-learn is by far the best library out there for machine learning, I think if you want an easy solution, you probably won't get around TensorFlow or PyTorch or equivalent, here, for your specific MLP problem unless you want to make your life extra hard :P (seriously, you can pick up any of the two in about an hour and have your MLPRegressor up and running so that you can then experiment with your cost function).
> 
> Best,
> Sebastian
> 
> > On Sep 11, 2017, at 6:13 PM, Thomas Evangelidis <tevang3 at gmail.com> wrote:
> >
> > Greetings,
> >
> > I know this is a recurrent question, but I would like to use my own loss function either in a MLPRegressor or in an SVR. For the MLPRegressor case so far my conclusion was that it is not possible unless you modify the source code. On the other hand, for the SVR I was looking at setting custom kernel functions. But I am not sure if this is the same thing. Could someone please clarify this to me? Finally, I read about the "scoring" parameter is cross-validation, but this is just to select a Regressor that has been trained already with the default loss function, so it would be harder to find one that minimizes my own loss function.
> >
> > For the record, my loss function is the centered root mean square error.
> >
> > Thanks in advance for any advice.
> >
> >
> >
> > --
> > ======================================================================
> > Dr Thomas Evangelidis
> > Post-doctoral Researcher
> > CEITEC - Central European Institute of Technology
> > Masaryk University
> > Kamenice 5/A35/2S049,
> > 62500 Brno, Czech Republic
> >
> > email: tevang at pharm.uoa.gr
> >               tevang3 at gmail.com
> >
> > website: https://sites.google.com/site/thomasevangelidishomepage/
> >
> >
> > _______________________________________________
> > scikit-learn mailing list
> > scikit-learn at python.org
> > https://mail.python.org/mailman/listinfo/scikit-learn
> 
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
> 
> 
> 
> -- 
> ======================================================================
> Dr Thomas Evangelidis
> Post-doctoral Researcher
> CEITEC - Central European Institute of Technology
> Masaryk University
> Kamenice 5/A35/2S049, 
> 62500 Brno, Czech Republic 
> 
> email: tevang at pharm.uoa.gr
>          	tevang3 at gmail.com
> 
> website: https://sites.google.com/site/thomasevangelidishomepage/
> 
> 
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn