On Wed, 23 Nov 2016 at 17:31 Joel Nothman <joel.nothman@gmail.com> wrote:
Raghav's example of
[{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver': 'sgd'}, {'solver': 'adam'}]
was not correct.
Should be
[{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver': ['sgd']}, {'solver': ['adam']}]
(Note all values of dicts are lists)
Ah, thanks! (Just ran into an error as it started iterating over the "sgd".)
On 23 November 2016 at 22:52, Jaidev Deshpande <deshpande.jaidev@gmail.com
wrote:
On Wed, 23 Nov 2016 at 16:29 Raghav R V <ragvrv@gmail.com> wrote:
Hi!
What you could do is specify lists of dicts to group the parameters which apply together in one dict...
[{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver': 'sgd'}, {'solver': 'adam'}]
```py from sklearn.neural_network import MLPClassifier from sklearn.model_selection import GridSearchCV from sklearn.datasets import make_classification
from pandas import DataFrame
X, y = make_classification(random_state=42)
gs = GridSearchCV(MLPClassifier(random_state=42), param_grid=[{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver': ['sgd',]}, {'solver': ['adam',]}])
DataFrame(gs.fit(X, y).cv_results_) ```
Would give
[image: image.png]
HTH :)
Haha, this is perfect. I didn't know you could pass a list of dicts to param_grid.
Thanks!
On Wed, Nov 23, 2016 at 11:15 AM, Jaidev Deshpande < deshpande.jaidev@gmail.com> wrote:
Hi,
Sometimes when using GridSearchCV, I realize that in the grid there are certain combinations of hyperparameters that are either incompatible or redundant. For example, when using an MLP, if I specify the following grid:
grid = {'solver': ['sgd', 'adam'], 'learning_rate': ['constant', 'invscaling', 'adaptive']}
then it yields the following ParameterGrid:
[{'learning_rate': 'constant', 'solver': 'sgd'}, {'learning_rate': 'constant', 'solver': 'adam'}, {'learning_rate': 'invscaling', 'solver': 'sgd'}, {'learning_rate': 'invscaling', 'solver': 'adam'}, {'learning_rate': 'adaptive', 'solver': 'sgd'}, {'learning_rate': 'adaptive', 'solver': 'adam'}]
Now, three of these are redundant, since learning_rate is used only for the sgd solver. Ideally I'd like to specify these cases upfront, and for that I have a simple hack ( https://github.com/jaidevd/jarvis/blob/master/jarvis/cross_validation.py#L38). Using that yields a ParameterGrid as follows:
[{'learning_rate': 'constant', 'solver': 'adam'}, {'learning_rate': 'invscaling', 'solver': 'adam'}, {'learning_rate': 'adaptive', 'solver': 'adam'}]
which is then simply removed from the original ParameterGrid.
I wonder if there's a simpler way of doing this. Would it help if we had an additional parameter (something like "grid_exceptions") in GridSearchCV, which would remove these dicts from the list of parameters?
Thanks
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
-- Raghav RV https://github.com/raghavrv
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
