[scikit-learn] control value range of MLPRegressor predictions

[Sebastian Raschka]

With clipping, I mean thresholding the output, e.g., via sth like
min/max(some_constant, actual_output)
or like in an leaky relu:
min/max(some_constant * 0.001, actual_output)

Alternatively, you could use an sigmoidal function (something like tanh but with a larger co-domain) as the output unit, but I am not sure the MLPRegressor allows that. In that case, you probably want to implement the MLP regressor yourself (e.g., via TensorFlow or PyTorch) to have some room for experimentation with your output units.

Best,
Sebastian




> On Sep 10, 2017, at 4:43 PM, Thomas Evangelidis <tevang3 at gmail.com> wrote:
> 
> 
> 
> On 10 September 2017 at 22:03, Sebastian Raschka <se.raschka at gmail.com> wrote:
> You could normalize the outputs (e.g., via min-max scaling). However, I think the more intuitive way would be to clip the predictions. E.g., say you are predicting house prices, it probably makes no sense to have a negative prediction, so you would clip the output at some value  >0$
> 
> 
> ​By clipping you mean discarding the predictors that give values below/above the threshold? 
> 
>  
> PS: -820 and -800 sounds a bit extreme if your training data is in a -5 to -9 range. Is your training data from a different population then the one you use for testing/making predictions? Or maybe it's just an extreme case of overfitting.
> 
> 
> ​It is from the same population, but the training sets I use are very small (6-32 observations), so it must be over-fitting. We had that discussion in the past here, yet in practice I get good correlations with the experimental values using MLPRegressors.​
> 
>  
> Best,
> Sebastian
> 
> 
> > On Sep 10, 2017, at 3:13 PM, Thomas Evangelidis <tevang3 at gmail.com> wrote:
> >
> > Greetings,
> >
> > Is there any way to force the MLPRegressor to make predictions in the same value range as the training data? For example, if the training data range between -5 and -9, I don't want the predictions to range between -820 and -800. In fact, some times I get anti-correlated predictions, for example between 800 and 820 and I have to change the sign in order to calculate correlations with experimental values. Is there a way to control the value range explicitly or implicitly (by post-processing the predictions)?
> >
> > thanks
> > Thomas
> >
> >
> > --
> > ======================================================================
> > Dr Thomas Evangelidis
> > Post-doctoral Researcher
> > CEITEC - Central European Institute of Technology
> > Masaryk University
> > Kamenice 5/A35/2S049,
> > 62500 Brno, Czech Republic
> >
> > email: tevang at pharm.uoa.gr
> >               tevang3 at gmail.com
> >
> > website: https://sites.google.com/site/thomasevangelidishomepage/
> >
> >
> > _______________________________________________
> > scikit-learn mailing list
> > scikit-learn at python.org
> > https://mail.python.org/mailman/listinfo/scikit-learn
> 
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
> 
> 
> 
> -- 
> ======================================================================
> Dr Thomas Evangelidis
> Post-doctoral Researcher
> CEITEC - Central European Institute of Technology
> Masaryk University
> Kamenice 5/A35/2S049, 
> 62500 Brno, Czech Republic 
> 
> email: tevang at pharm.uoa.gr
>          	tevang3 at gmail.com
> 
> website: https://sites.google.com/site/thomasevangelidishomepage/
> 
> 
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn