<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Wed, 23 Nov 2016 at 16:29 Raghav R V <<a href="mailto:ragvrv@gmail.com">ragvrv@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg"><div class="gmail_msg">Hi!</div><div class="gmail_msg"><br class="gmail_msg"></div>What you could do is specify lists of dicts to group the parameters which apply together in one dict...<div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">[{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver': 'sgd'}, {'solver': 'adam'}]</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">```py</div><div class="gmail_msg"><div class="gmail_msg">from sklearn.neural_network import MLPClassifier</div><div class="gmail_msg">from sklearn.model_selection import GridSearchCV</div><div class="gmail_msg">from sklearn.datasets import make_classification</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">from pandas import DataFrame</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">X, y = make_classification(random_state=42)</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">gs = GridSearchCV(MLPClassifier(random_state=42),</div><div class="gmail_msg"> param_grid=[{'learning_rate': ['constant', 'invscaling', 'adaptive'],</div><div class="gmail_msg"> 'solver': ['sgd',]},</div><div class="gmail_msg"> {'solver': ['adam',]}])</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">DataFrame(gs.fit(X, y).cv_results_)</div></div><div class="gmail_msg">```</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">Would give</div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg"><img src="cid:ii_15890d486a7f0816" alt="image.png" class="" style="max-width: 100%;"><br class="gmail_msg"></div><div class="gmail_msg"><br class="gmail_msg"></div><div class="gmail_msg">HTH :)</div></div></blockquote><div><br></div><div>Haha, this is perfect. I didn't know you could pass a list of dicts to param_grid.</div><div><br></div><div>Thanks!</div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><br class="gmail_msg"><div class="gmail_quote gmail_msg"></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg">On Wed, Nov 23, 2016 at 11:15 AM, Jaidev Deshpande <span dir="ltr" class="gmail_msg"><<a href="mailto:deshpande.jaidev@gmail.com" class="gmail_msg" target="_blank">deshpande.jaidev@gmail.com</a>></span> wrote:<br class="gmail_msg"></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr" class="gmail_msg"><span style="color:rgb(33,33,33);font-size:13px" class="gmail_msg">Hi,</span><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px">Sometimes when using GridSearchCV, I realize that in the grid there are certain combinations of hyperparameters that are either incompatible or redundant. For example, when using an MLP, if I specify the following grid:</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px">grid = {'solver': ['sgd', 'adam'], 'learning_rate': ['constant', 'invscaling', 'adaptive']}<br class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px">then it yields the following ParameterGrid:</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px"><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg">[{'learning_rate': 'constant', 'solver': 'sgd'},</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"> {'learning_rate': 'constant', 'solver': 'adam'},</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"> {'learning_rate': 'invscaling', 'solver': 'sgd'},</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"> {'learning_rate': 'invscaling', 'solver': 'adam'},</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"> {'learning_rate': 'adaptive', 'solver': 'sgd'},</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"> {'learning_rate': 'adaptive', 'solver': 'adam'}]</div></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px">Now, three of these are redundant, since learning_rate is used only for the sgd solver. Ideally I'd like to specify these cases upfront, and for that I have a simple hack (<a href="https://github.com/jaidevd/jarvis/blob/master/jarvis/cross_validation.py#L38" class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" target="_blank">https://github.com/jaidevd/jarvis/blob/master/jarvis/cross_validation.py#L38</a>). Using that yields a ParameterGrid as follows:</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px"><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg">[{'learning_rate': 'constant', 'solver': 'adam'},</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"> {'learning_rate': 'invscaling', 'solver': 'adam'},</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"> {'learning_rate': 'adaptive', 'solver': 'adam'}]</div></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px">which is then simply removed from the original ParameterGrid.</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px">I wonder if there's a simpler way of doing this. Would it help if we had an additional parameter (something like "grid_exceptions") in GridSearchCV, which would remove these dicts from the list of parameters?</div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px"><br class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg"></div><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777m_1665174164762036910gmail_msg gmail_msg" style="color:rgb(33,33,33);font-size:13px">Thanks</div></div>
<br class="gmail_msg"></blockquote></div></div></div><div dir="ltr" class="gmail_msg"><div class="gmail_extra gmail_msg"><div class="gmail_quote gmail_msg"><blockquote class="gmail_quote gmail_msg" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">_______________________________________________<br class="gmail_msg">
scikit-learn mailing list<br class="gmail_msg">
<a href="mailto:scikit-learn@python.org" class="gmail_msg" target="_blank">scikit-learn@python.org</a><br class="gmail_msg">
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" class="gmail_msg" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br class="gmail_msg">
<br class="gmail_msg"></blockquote></div><br class="gmail_msg"><br clear="all" class="gmail_msg"><div class="gmail_msg"><br class="gmail_msg"></div>-- <br class="gmail_msg"><div class="m_3889491088195474880m_1746681790862895468gmail-m_-9096476856000922777gmail_signature gmail_msg"><div dir="ltr" class="gmail_msg"><div style="font-size:small" class="gmail_msg">Raghav RV</div><div style="font-size:small" class="gmail_msg"><a href="https://github.com/raghavrv" style="color:rgb(17,85,204)" class="gmail_msg" target="_blank">https://github.com/raghavrv</a></div><div class="gmail_msg"><br class="gmail_msg"></div></div></div>
</div></div>
_______________________________________________<br class="gmail_msg">
scikit-learn mailing list<br class="gmail_msg">
<a href="mailto:scikit-learn@python.org" class="gmail_msg" target="_blank">scikit-learn@python.org</a><br class="gmail_msg">
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" class="gmail_msg" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br class="gmail_msg">
</blockquote></div></div>