[scikit-learn] meta-estimator for multiple MLPRegressor

[Jacob Schreiber]

Even with a single layer with 10 neurons you're still trying to train over
6000 parameters using ~30 samples. Dropout is a concept common in neural
networks, but doesn't appear to be in sklearn's implementation of MLPs.
Early stopping based on validation performance isn't an "extra" step for
reducing overfitting, it's basically a required step for neural networks.
It seems like you have a validation sample of ~6 datapoints.. I'm still
very skeptical of that giving you proper results for a complex model. Will
this larger dataset be of exactly the same data? Just taking another
unrelated dataset and showing that a MLP can learn it doesn't mean it will
work for your specific data. Can you post the actual results from using
LASSO, RandomForestRegressor, GradientBoostingRegressor, and MLP?

On Mon, Jan 9, 2017 at 4:21 PM, Stuart Reynolds <stuart at stuartreynolds.net>
wrote:

> If you dont have a large dataset, you can still do leave one out cross
> validation.
>
> On Mon, Jan 9, 2017 at 3:42 PM Thomas Evangelidis <tevang3 at gmail.com>
> wrote:
>
>>
>> Jacob & Sebastian,
>>
>> I think the best way to find out if my modeling approach works is to find
>> a larger dataset, split it into two parts, the first one will be used as
>> training/cross-validation set and the second as a test set, like in a real
>> case scenario.
>>
>> Regarding the MLPRegressor regularization, below is my optimum setup:
>>
>> MLPRegressor(random_state=random_state, max_iter=400,
>> early_stopping=True, validation_fraction=0.2, alpha=10,
>> hidden_layer_sizes=(10,))
>>
>>
>> This means only one hidden layer with maximum 10 neurons, alpha=10 for L2
>> regularization and early stopping to terminate training if validation score
>> is not improving. I think this is a quite simple model. My final predictor
>> is an SVR that combines 2 MLPRegressors, each one trained with different
>> types of input data.
>>
>> @Sebastian
>> You have mentioned dropout again but I could not find it in the docs:
>> http://scikit-learn.org/stable/modules/generated/sklearn.neural_network.
>> MLPRegressor.html#sklearn.neural_network.MLPRegressor
>>
>> Maybe you are referring to another MLPRegressor implementation? I have
>> seen a while ago another implementation you had on github. Can you clarify
>> which one you recommend and why?
>>
>>
>> Thank you both of you for your hints!
>>
>> best
>> Thomas
>>
>>
>>
>> --
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> ======================================================================
>>
>>
>> Thomas Evangelidis
>>
>>
>> Research Specialist
>> CEITEC - Central European Institute of Technology
>> Masaryk University
>> Kamenice 5/A35/1S081,
>> 62500 Brno, Czech Republic
>>
>> email: tevang at pharm.uoa.gr
>>
>>
>>           tevang3 at gmail.com
>>
>>
>>
>> website:
>>
>> https://sites.google.com/site/thomasevangelidishomepage/
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>>
>> scikit-learn mailing list
>>
>> scikit-learn at python.org
>>
>> https://mail.python.org/mailman/listinfo/scikit-learn
>>
>>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20170109/6588a051/attachment-0001.html>