On Wed, 23 Nov 2016 at 19:05 Roman Yurchak <rth.yurchak@gmail.com> wrote:
Hi Jaidev,
well, `param_grid` in GridSearchCV can also be a list of dictionaries, so you could directly specify the cases you are interested in (instead of the full grid - exceptions), which might be simpler?
Actually now that I think of it, I don't know if it will be necessarily simpler. What if I have a massive grid and only few exceptions? Enumerating the complement of that small subset would be much more expensive than specifying the exceptions. What do you think?
On 23/11/16 11:15, Jaidev Deshpande wrote:
Hi,
Sometimes when using GridSearchCV, I realize that in the grid there are certain combinations of hyperparameters that are either incompatible or redundant. For example, when using an MLP, if I specify the following grid:
grid = {'solver': ['sgd', 'adam'], 'learning_rate': ['constant', 'invscaling', 'adaptive']}
then it yields the following ParameterGrid:
[{'learning_rate': 'constant', 'solver': 'sgd'}, {'learning_rate': 'constant', 'solver': 'adam'}, {'learning_rate': 'invscaling', 'solver': 'sgd'}, {'learning_rate': 'invscaling', 'solver': 'adam'}, {'learning_rate': 'adaptive', 'solver': 'sgd'}, {'learning_rate': 'adaptive', 'solver': 'adam'}]
Now, three of these are redundant, since learning_rate is used only for the sgd solver. Ideally I'd like to specify these cases upfront, and for that I have a simple hack ( https://github.com/jaidevd/jarvis/blob/master/jarvis/cross_validation.py#L38 ). Using that yields a ParameterGrid as follows:
[{'learning_rate': 'constant', 'solver': 'adam'}, {'learning_rate': 'invscaling', 'solver': 'adam'}, {'learning_rate': 'adaptive', 'solver': 'adam'}]
which is then simply removed from the original ParameterGrid.
I wonder if there's a simpler way of doing this. Would it help if we had an additional parameter (something like "grid_exceptions") in GridSearchCV, which would remove these dicts from the list of parameters?
Thanks
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn