[scikit-learn] custom loss function
Thomas Evangelidis
tevang3 at gmail.com
Wed Sep 13 13:18:39 EDT 2017
Thanks again for the clarifications Sebastian!
Keras has a Scikit-learn API with the KeraRegressor which implements the
Scikit-Learn MLPRegressor interface:
https://keras.io/scikit-learn-api/
Is it possible to change the loss function in KerasRegressor? I don't have
time right now to experiment with hyperparameters of new ANN architectures.
I am in urgent need to reproduce in Keras the results obtained with
MLPRegressor and the set of hyperparameters that I have optimized for my
problem and later change the loss function.
On 13 September 2017 at 18:14, Sebastian Raschka <se.raschka at gmail.com>
wrote:
> > What about the SVR? Is it possible to change the loss function there?
>
> Here you would have the same problem; SVR is a constrained optimization
> problem and you would have to change the calculation of the loss gradient
> then. Since SVR is a "1-layer" neural net, if you change the cost function
> to something else, it's not really a SVR anymore.
>
>
> > Could you please clarify what the "x" and "x'" parameters in the default
> Kernel functions mean? Is "x" a NxM array, where N is the number of
> observations and M the number of features?
>
> Both x and x' should be denoting training examples. The kernel matrix is
> symmetric (N x N).
>
>
>
> Best,
> Sebastian
>
> > On Sep 13, 2017, at 5:25 AM, Thomas Evangelidis <tevang3 at gmail.com>
> wrote:
> >
> > Thanks Sebastian. Exploring Tensorflow capabilities was in my TODO list,
> but now it's in my immediate plans.
> > What about the SVR? Is it possible to change the loss function there?
> Could you please clarify what the "x" and "x'" parameters in the default
> Kernel functions mean? Is "x" a NxM array, where N is the number of
> observations and M the number of features?
> >
> > http://scikit-learn.org/stable/modules/svm.html#kernel-functions
> >
> >
> >
> > On 12 September 2017 at 00:37, Sebastian Raschka <se.raschka at gmail.com>
> wrote:
> > Hi Thomas,
> >
> > > For the MLPRegressor case so far my conclusion was that it is not
> possible unless you modify the source code.
> >
> > Also, I suspect that this would be non-trivial. I haven't looked to
> closely at how the MLPClassifier/MLPRegressor are implemented but since you
> perform the weight updates based on the gradient of the cost function wrt
> the weights, the modification would be non-trivial if the partial
> derivatives are not computed based on some autodiff implementation -- you
> would have to edit all the partial d's along the backpropagation up to the
> first hidden layer. While I think that scikit-learn is by far the best
> library out there for machine learning, I think if you want an easy
> solution, you probably won't get around TensorFlow or PyTorch or
> equivalent, here, for your specific MLP problem unless you want to make
> your life extra hard :P (seriously, you can pick up any of the two in about
> an hour and have your MLPRegressor up and running so that you can then
> experiment with your cost function).
> >
> > Best,
> > Sebastian
> >
> > > On Sep 11, 2017, at 6:13 PM, Thomas Evangelidis <tevang3 at gmail.com>
> wrote:
> > >
> > > Greetings,
> > >
> > > I know this is a recurrent question, but I would like to use my own
> loss function either in a MLPRegressor or in an SVR. For the MLPRegressor
> case so far my conclusion was that it is not possible unless you modify the
> source code. On the other hand, for the SVR I was looking at setting custom
> kernel functions. But I am not sure if this is the same thing. Could
> someone please clarify this to me? Finally, I read about the "scoring"
> parameter is cross-validation, but this is just to select a Regressor that
> has been trained already with the default loss function, so it would be
> harder to find one that minimizes my own loss function.
> > >
> > > For the record, my loss function is the centered root mean square
> error.
> > >
> > > Thanks in advance for any advice.
> > >
> > >
> > >
> > > --
> > > ======================================================================
> > > Dr Thomas Evangelidis
> > > Post-doctoral Researcher
> > > CEITEC - Central European Institute of Technology
> > > Masaryk University
> > > Kamenice 5/A35/2S049,
> > > 62500 Brno, Czech Republic
> > >
> > > email: tevang at pharm.uoa.gr
> > > tevang3 at gmail.com
> > >
> > > website: https://sites.google.com/site/thomasevangelidishomepage/
> > >
> > >
> > > _______________________________________________
> > > scikit-learn mailing list
> > > scikit-learn at python.org
> > > https://mail.python.org/mailman/listinfo/scikit-learn
> >
> > _______________________________________________
> > scikit-learn mailing list
> > scikit-learn at python.org
> > https://mail.python.org/mailman/listinfo/scikit-learn
> >
> >
> >
> > --
> > ======================================================================
> > Dr Thomas Evangelidis
> > Post-doctoral Researcher
> > CEITEC - Central European Institute of Technology
> > Masaryk University
> > Kamenice 5/A35/2S049,
> > 62500 Brno, Czech Republic
> >
> > email: tevang at pharm.uoa.gr
> > tevang3 at gmail.com
> >
> > website: https://sites.google.com/site/thomasevangelidishomepage/
> >
> >
> > _______________________________________________
> > scikit-learn mailing list
> > scikit-learn at python.org
> > https://mail.python.org/mailman/listinfo/scikit-learn
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
--
======================================================================
Dr Thomas Evangelidis
Post-doctoral Researcher
CEITEC - Central European Institute of Technology
Masaryk University
Kamenice 5/A35/2S049,
62500 Brno, Czech Republic
email: tevang at pharm.uoa.gr
tevang3 at gmail.com
website: https://sites.google.com/site/thomasevangelidishomepage/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20170913/6aeddad3/attachment-0001.html>
More information about the scikit-learn
mailing list