<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Wed, 23 Nov 2016 at 19:05 Roman Yurchak <<a href="mailto:rth.yurchak@gmail.com">rth.yurchak@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Jaidev,<br class="gmail_msg">
<br class="gmail_msg">
well, `param_grid` in GridSearchCV can also be a list of dictionaries,<br class="gmail_msg">
so you could directly specify the cases you are interested in (instead<br class="gmail_msg">
of the full grid - exceptions), which might be simpler?<br class="gmail_msg"></blockquote><div><br></div><div>Actually now that I think of it, I don't know if it will be necessarily simpler. What if I have a massive grid and only few exceptions? Enumerating the complement of that small subset would be much more expensive than specifying the exceptions.</div><div><br></div><div>What do you think?</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<br class="gmail_msg">
On 23/11/16 11:15, Jaidev Deshpande wrote:<br class="gmail_msg">
> Hi,<br class="gmail_msg">
><br class="gmail_msg">
> Sometimes when using GridSearchCV, I realize that in the grid there are<br class="gmail_msg">
> certain combinations of hyperparameters that are either incompatible or<br class="gmail_msg">
> redundant. For example, when using an MLP, if I specify the following grid:<br class="gmail_msg">
><br class="gmail_msg">
> grid = {'solver': ['sgd', 'adam'], 'learning_rate': ['constant',<br class="gmail_msg">
> 'invscaling', 'adaptive']}<br class="gmail_msg">
><br class="gmail_msg">
> then it yields the following ParameterGrid:<br class="gmail_msg">
><br class="gmail_msg">
> [{'learning_rate': 'constant', 'solver': 'sgd'},<br class="gmail_msg">
>  {'learning_rate': 'constant', 'solver': 'adam'},<br class="gmail_msg">
>  {'learning_rate': 'invscaling', 'solver': 'sgd'},<br class="gmail_msg">
>  {'learning_rate': 'invscaling', 'solver': 'adam'},<br class="gmail_msg">
>  {'learning_rate': 'adaptive', 'solver': 'sgd'},<br class="gmail_msg">
>  {'learning_rate': 'adaptive', 'solver': 'adam'}]<br class="gmail_msg">
><br class="gmail_msg">
> Now, three of these are redundant, since learning_rate is used only for<br class="gmail_msg">
> the sgd solver. Ideally I'd like to specify these cases upfront, and for<br class="gmail_msg">
> that I have a simple hack<br class="gmail_msg">
> (<a href="https://github.com/jaidevd/jarvis/blob/master/jarvis/cross_validation.py#L38" rel="noreferrer" class="gmail_msg" target="_blank">https://github.com/jaidevd/jarvis/blob/master/jarvis/cross_validation.py#L38</a>).<br class="gmail_msg">
> Using that yields a ParameterGrid as follows:<br class="gmail_msg">
><br class="gmail_msg">
> [{'learning_rate': 'constant', 'solver': 'adam'},<br class="gmail_msg">
>  {'learning_rate': 'invscaling', 'solver': 'adam'},<br class="gmail_msg">
>  {'learning_rate': 'adaptive', 'solver': 'adam'}]<br class="gmail_msg">
><br class="gmail_msg">
> which is then simply removed from the original ParameterGrid.<br class="gmail_msg">
><br class="gmail_msg">
> I wonder if there's a simpler way of doing this. Would it help if we had<br class="gmail_msg">
> an additional parameter (something like "grid_exceptions") in<br class="gmail_msg">
> GridSearchCV, which would remove these dicts from the list of parameters?<br class="gmail_msg">
><br class="gmail_msg">
> Thanks<br class="gmail_msg">
><br class="gmail_msg">
><br class="gmail_msg">
> _______________________________________________<br class="gmail_msg">
> scikit-learn mailing list<br class="gmail_msg">
> <a href="mailto:scikit-learn@python.org" class="gmail_msg" target="_blank">scikit-learn@python.org</a><br class="gmail_msg">
> <a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" class="gmail_msg" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br class="gmail_msg">
><br class="gmail_msg">
<br class="gmail_msg">
_______________________________________________<br class="gmail_msg">
scikit-learn mailing list<br class="gmail_msg">
<a href="mailto:scikit-learn@python.org" class="gmail_msg" target="_blank">scikit-learn@python.org</a><br class="gmail_msg">
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" class="gmail_msg" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br class="gmail_msg">
</blockquote></div></div>