Hi!

What you could do is specify lists of dicts to group the parameters which apply together in one dict...

[{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver': 'sgd'}, {'solver': 'adam'}]

```py
from sklearn.neural_network import MLPClassifier
from sklearn.model_selection import GridSearchCV
from sklearn.datasets import make_classification

from pandas import DataFrame

X, y = make_classification(random_state=42)

gs = GridSearchCV(MLPClassifier(random_state=42),
                  param_grid=[{'learning_rate': ['constant', 'invscaling', 'adaptive'],
                               'solver': ['sgd',]},
                              {'solver': ['adam',]}])

DataFrame(gs.fit(X, y).cv_results_)
```

Would give

Inline image 1

HTH :)

On Wed, Nov 23, 2016 at 11:15 AM, Jaidev Deshpande <deshpande.jaidev@gmail.com> wrote:
Hi,

Sometimes when using GridSearchCV, I realize that in the grid there are certain combinations of hyperparameters that are either incompatible or redundant. For example, when using an MLP, if I specify the following grid:

grid = {'solver': ['sgd', 'adam'], 'learning_rate': ['constant', 'invscaling', 'adaptive']}

then it yields the following ParameterGrid:

[{'learning_rate': 'constant', 'solver': 'sgd'},
 {'learning_rate': 'constant', 'solver': 'adam'},
 {'learning_rate': 'invscaling', 'solver': 'sgd'},
 {'learning_rate': 'invscaling', 'solver': 'adam'},
 {'learning_rate': 'adaptive', 'solver': 'sgd'},
 {'learning_rate': 'adaptive', 'solver': 'adam'}]

Now, three of these are redundant, since learning_rate is used only for the sgd solver. Ideally I'd like to specify these cases upfront, and for that I have a simple hack (https://github.com/jaidevd/jarvis/blob/master/jarvis/cross_validation.py#L38). Using that yields a ParameterGrid as follows:

[{'learning_rate': 'constant', 'solver': 'adam'},
 {'learning_rate': 'invscaling', 'solver': 'adam'},
 {'learning_rate': 'adaptive', 'solver': 'adam'}]

which is then simply removed from the original ParameterGrid.

I wonder if there's a simpler way of doing this. Would it help if we had an additional parameter (something like "grid_exceptions") in GridSearchCV, which would remove these dicts from the list of parameters?

Thanks

_______________________________________________
scikit-learn mailing list
scikit-learn@python.org
https://mail.python.org/mailman/listinfo/scikit-learn




--
Raghav RV
https://github.com/raghavrv