[scikit-learn] Specifying exceptions to ParameterGrid

Raghav R V ragvrv at gmail.com
Wed Nov 23 05:57:14 EST 2016


Hi!

What you could do is specify lists of dicts to group the parameters which
apply together in one dict...

[{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver':
'sgd'}, {'solver': 'adam'}]

```py
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.datasets import make_classification

from pandas import DataFrame

X, y = make_classification(random_state=42)

gs = GridSearchCV(MLPClassifier(random_state=42),
                  param_grid=[{'learning_rate': ['constant', 'invscaling',
'adaptive'],
                               'solver': ['sgd',]},
                              {'solver': ['adam',]}])

DataFrame(gs.fit(X, y).cv_results_)
```

Would give

[image: Inline image 1]

HTH :)

On Wed, Nov 23, 2016 at 11:15 AM, Jaidev Deshpande <
deshpande.jaidev at gmail.com> wrote:

> Hi,
>
> Sometimes when using GridSearchCV, I realize that in the grid there are
> certain combinations of hyperparameters that are either incompatible or
> redundant. For example, when using an MLP, if I specify the following grid:
>
> grid = {'solver': ['sgd', 'adam'], 'learning_rate': ['constant',
> 'invscaling', 'adaptive']}
>
> then it yields the following ParameterGrid:
>
> [{'learning_rate': 'constant', 'solver': 'sgd'},
>  {'learning_rate': 'constant', 'solver': 'adam'},
>  {'learning_rate': 'invscaling', 'solver': 'sgd'},
>  {'learning_rate': 'invscaling', 'solver': 'adam'},
>  {'learning_rate': 'adaptive', 'solver': 'sgd'},
>  {'learning_rate': 'adaptive', 'solver': 'adam'}]
>
> Now, three of these are redundant, since learning_rate is used only for
> the sgd solver. Ideally I'd like to specify these cases upfront, and for
> that I have a simple hack (https://github.com/jaidevd/ja
> rvis/blob/master/jarvis/cross_validation.py#L38). Using that yields a
> ParameterGrid as follows:
>
> [{'learning_rate': 'constant', 'solver': 'adam'},
>  {'learning_rate': 'invscaling', 'solver': 'adam'},
>  {'learning_rate': 'adaptive', 'solver': 'adam'}]
>
> which is then simply removed from the original ParameterGrid.
>
> I wonder if there's a simpler way of doing this. Would it help if we had
> an additional parameter (something like "grid_exceptions") in GridSearchCV,
> which would remove these dicts from the list of parameters?
>
> Thanks
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
>


-- 
Raghav RV
https://github.com/raghavrv
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20161123/6f6039b2/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 52450 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20161123/6f6039b2/attachment-0001.png>


More information about the scikit-learn mailing list