[scikit-learn] Specifying exceptions to ParameterGrid

[Roman Yurchak]

Hi Jaidev,

well, `param_grid` in GridSearchCV can also be a list of dictionaries,
so you could directly specify the cases you are interested in (instead
of the full grid - exceptions), which might be simpler?

On 23/11/16 11:15, Jaidev Deshpande wrote:
> Hi,
> 
> Sometimes when using GridSearchCV, I realize that in the grid there are
> certain combinations of hyperparameters that are either incompatible or
> redundant. For example, when using an MLP, if I specify the following grid:
> 
> grid = {'solver': ['sgd', 'adam'], 'learning_rate': ['constant',
> 'invscaling', 'adaptive']}
> 
> then it yields the following ParameterGrid:
> 
> [{'learning_rate': 'constant', 'solver': 'sgd'},
>  {'learning_rate': 'constant', 'solver': 'adam'},
>  {'learning_rate': 'invscaling', 'solver': 'sgd'},
>  {'learning_rate': 'invscaling', 'solver': 'adam'},
>  {'learning_rate': 'adaptive', 'solver': 'sgd'},
>  {'learning_rate': 'adaptive', 'solver': 'adam'}]
> 
> Now, three of these are redundant, since learning_rate is used only for
> the sgd solver. Ideally I'd like to specify these cases upfront, and for
> that I have a simple hack
> (https://github.com/jaidevd/jarvis/blob/master/jarvis/cross_validation.py#L38).
> Using that yields a ParameterGrid as follows:
> 
> [{'learning_rate': 'constant', 'solver': 'adam'},
>  {'learning_rate': 'invscaling', 'solver': 'adam'},
>  {'learning_rate': 'adaptive', 'solver': 'adam'}]
> 
> which is then simply removed from the original ParameterGrid.
> 
> I wonder if there's a simpler way of doing this. Would it help if we had
> an additional parameter (something like "grid_exceptions") in
> GridSearchCV, which would remove these dicts from the list of parameters?
> 
> Thanks
> 
> 
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>