[scikit-learn] custom loss function in RandomForestRegressor

Guillaume Lemaître g.lemaitre58 at gmail.com
Thu Feb 15 12:50:38 EST 2018

The ClassificationCriterion and RegressionCriterion are now exposed in the
_criterion.pxd. It will allow you to create your own criterion.
So you can write your own Criterion with a given loss by implementing the
methods which are required in the trees.
Then you can pass an instance of this criterion to the tree and it should

On 15 February 2018 at 18:37, Thomas Evangelidis <tevang3 at gmail.com> wrote:

> Greetings,
> The feature importance calculated by the RandomForest implementation is a
> very useful feature. I personally use it to select the best features
> because it is simple and fast, and then I train MLPRegressors. The
> limitation of this approach is that although I can control the loss
> function of the MLPRegressor (I have modified scikit-learn's implementation
> to accept an arbitrary loss function), I cannot do the same with
> RandomForestRegressor, and hence I have to rely on 'mse' which is not in
> accordance with the loss functions I use in MLPs. Today I was looking at
> the _criterion.pyx file:
> https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/tree/_
> criterion.pyx
> However, the code is in Cython and I find it hard to follow. I know that
> for Regression the relevant class are Criterion(),
> RegressionCriterion(Criterion), and MSE(RegressionCriterion). My question
> is: is it possible to write a class that takes an arbitrary function
> "loss(predictions, targets)" to calculate the loss and impurity of the
> nodes?
> thanks,
> Thomas
> --
> ======================================================================
> Dr Thomas Evangelidis
> Post-doctoral Researcher
> CEITEC - Central European Institute of Technology
> Masaryk University
> Kamenice 5/A35/2S049,
> 62500 Brno, Czech Republic
> email: tevang at pharm.uoa.gr
>           tevang3 at gmail.com
> website: https://sites.google.com/site/thomasevangelidishomepage/
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn

Guillaume Lemaitre
INRIA Saclay - Parietal team
Center for Data Science Paris-Saclay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20180215/c25db476/attachment-0001.html>

More information about the scikit-learn mailing list