[scikit-learn] Specifying exceptions to ParameterGrid

[Raghav R V]

On Wed, Nov 23, 2016 at 12:59 PM, Joel Nothman <joel.nothman at gmail.com>
wrote:

> Raghav's example of
>
>
> [{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver':
> 'sgd'}, {'solver': 'adam'}]
>
> was not correct.
>

Oops sorry. Ah I ran into that, corrected it in the snipped but forgot to
update the line before the snippet... :)


> Should be
>
>
> [{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver':
> ['sgd']}, {'solver': ['adam']}]
>
> (Note all values of dicts are lists)
>
> On 23 November 2016 at 22:52, Jaidev Deshpande <deshpande.jaidev at gmail.com
> > wrote:
>
>>
>>
>> On Wed, 23 Nov 2016 at 16:29 Raghav R V <ragvrv at gmail.com> wrote:
>>
>>> Hi!
>>>
>>> What you could do is specify lists of dicts to group the parameters
>>> which apply together in one dict...
>>>
>>> [{'learning_rate': ['constant', 'invscaling', 'adaptive'], 'solver':
>>> 'sgd'}, {'solver': 'adam'}]
>>>
>>> ```py
>>> from sklearn.neural_network import MLPClassifier
>>> from sklearn.model_selection import GridSearchCV
>>> from sklearn.datasets import make_classification
>>>
>>> from pandas import DataFrame
>>>
>>> X, y = make_classification(random_state=42)
>>>
>>> gs = GridSearchCV(MLPClassifier(random_state=42),
>>>                   param_grid=[{'learning_rate': ['constant',
>>> 'invscaling', 'adaptive'],
>>>                                'solver': ['sgd',]},
>>>                               {'solver': ['adam',]}])
>>>
>>> DataFrame(gs.fit(X, y).cv_results_)
>>> ```
>>>
>>> Would give
>>>
>>> [image: image.png]
>>>
>>> HTH :)
>>>
>>
>> Haha, this is perfect. I didn't know you could pass a list of dicts to
>> param_grid.
>>
>> Thanks!
>>
>>
>>>
>>> On Wed, Nov 23, 2016 at 11:15 AM, Jaidev Deshpande <
>>> deshpande.jaidev at gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> Sometimes when using GridSearchCV, I realize that in the grid there are
>>> certain combinations of hyperparameters that are either incompatible or
>>> redundant. For example, when using an MLP, if I specify the following grid:
>>>
>>> grid = {'solver': ['sgd', 'adam'], 'learning_rate': ['constant',
>>> 'invscaling', 'adaptive']}
>>>
>>> then it yields the following ParameterGrid:
>>>
>>> [{'learning_rate': 'constant', 'solver': 'sgd'},
>>>  {'learning_rate': 'constant', 'solver': 'adam'},
>>>  {'learning_rate': 'invscaling', 'solver': 'sgd'},
>>>  {'learning_rate': 'invscaling', 'solver': 'adam'},
>>>  {'learning_rate': 'adaptive', 'solver': 'sgd'},
>>>  {'learning_rate': 'adaptive', 'solver': 'adam'}]
>>>
>>> Now, three of these are redundant, since learning_rate is used only for
>>> the sgd solver. Ideally I'd like to specify these cases upfront, and for
>>> that I have a simple hack (https://github.com/jaidevd/ja
>>> rvis/blob/master/jarvis/cross_validation.py#L38). Using that yields a
>>> ParameterGrid as follows:
>>>
>>> [{'learning_rate': 'constant', 'solver': 'adam'},
>>>  {'learning_rate': 'invscaling', 'solver': 'adam'},
>>>  {'learning_rate': 'adaptive', 'solver': 'adam'}]
>>>
>>> which is then simply removed from the original ParameterGrid.
>>>
>>> I wonder if there's a simpler way of doing this. Would it help if we had
>>> an additional parameter (something like "grid_exceptions") in GridSearchCV,
>>> which would remove these dicts from the list of parameters?
>>>
>>> Thanks
>>>
>>> _______________________________________________
>>> scikit-learn mailing list
>>> scikit-learn at python.org
>>> https://mail.python.org/mailman/listinfo/scikit-learn
>>>
>>>
>>>
>>>
>>> --
>>> Raghav RV
>>> https://github.com/raghavrv
>>>
>>> _______________________________________________
>>> scikit-learn mailing list
>>> scikit-learn at python.org
>>> https://mail.python.org/mailman/listinfo/scikit-learn
>>>
>>
>> _______________________________________________
>> scikit-learn mailing list
>> scikit-learn at python.org
>> https://mail.python.org/mailman/listinfo/scikit-learn
>>
>>
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn
>
>


-- 
Raghav RV
https://github.com/raghavrv
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20161123/4aa9bd41/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image.png
Type: image/png
Size: 52450 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20161123/4aa9bd41/attachment-0001.png>