<div dir="ltr">Hi Hugo,<div><br></div><div>Andrew's approach -- using a list of dicts to specify multiple parameter grids -- is the correct one.</div><div><br></div><div>However, Andrew, you don't need to include parameters that will be ignored into your parameter grid. The following will be effectively the same:</div><div><br></div><div><div style="font-size:12.8px">params = [{'kernel':['poly'],'degree':[1,2,3],'gamma':[1/p,1,2],'coef0':[-1,0,1]},</div><div style="font-size:12.8px"><span style="white-space:pre-wrap"> </span>{'kernel':['rbf'],'gamma':[1/p,1,2]},</div><div style="font-size:12.8px"><span style="white-space:pre-wrap"> </span>{'kernel':['sigmoid'],'gamma':[1/p,1,2],'coef0':[-1,0,1]}<span style="font-size:12.8px">]</span></div></div><div><br></div><div>Joel</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 27 June 2016 at 20:59, Andrew Howe <span dir="ltr"><<a href="mailto:ahowe42@gmail.com" target="_blank">ahowe42@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">I did something similar where I was using GridSearchCV over different kernel functions for SVM and not all kernel functions use the same parameters. For example, the <b>degree</b> parameter is only used by the <b>poly</b> kernel.<div><br><div><div>from sklearn import svm</div><div>from sklearn import cross_validation</div><div>from sklearn import grid_search</div></div><div><br></div><div><div>params = [{'kernel':['poly'],'degree':[1,2,3],'gamma':[1/p,1,2],'coef0':[-1,0,1]},\</div><div><span style="white-space:pre-wrap"> </span>{'kernel':['rbf'],'gamma':[1/p,1,2],'degree':[3],'coef0':[0]},\</div><div><span style="white-space:pre-wrap"> </span>{'kernel':['sigmoid'],'gamma':[1/p,1,2],'coef0':[-1,0,1],'degree':[3]}]</div><div>GSC = grid_search.GridSearchCV(estimator = svm.SVC(), param_grid = params,\</div><div> cv = cvrand, n_jobs = -1)</div></div><div><br></div><div>This worked in this instance because the svm.SVC() object only passes parameters to the kernel functions as needed: </div></div><div><img src="cid:ii_155918218974d2c7" alt="Inline image 1" width="472" height="177"><br></div><div><br></div><div>Hence, even though my list of dicts includes all three parameters for all types of kernels I used, they were selectively ignored. I'm not sure about parameters for the distance metrics for the KNN object, but it's a good bet it works the same way.</div><div><br></div><div>Andrew</div></div><div class="gmail_extra"><br clear="all"><div><div data-smartmail="gmail_signature"><div dir="ltr"><~~~~~~~~~~~~~~~~~~~~~~~~~~~><br>J. Andrew Howe, PhD<div>Editor-in-Chief, European Journal of Mathematical Sciences<br>Executive Editor, European Journal of Pure and Applied Mathematics<br><a href="http://www.andrewhowe.com" target="_blank">www.andrewhowe.com</a></div><div><a href="http://www.linkedin.com/in/ahowe42" style="color:rgb(17,85,204)" target="_blank">http://www.linkedin.com/in/ahowe42</a></div><div><a href="https://www.researchgate.net/profile/John_Howe12/" target="_blank">https://www.researchgate.net/profile/John_Howe12/</a><br>I live to learn, so I can learn to live. - me<br><~~~~~~~~~~~~~~~~~~~~~~~~~~~></div></div></div></div><div><div class="h5">
<br><div class="gmail_quote">On Mon, Jun 27, 2016 at 1:27 PM, Hugo Ferreira <span dir="ltr"><<a href="mailto:hmf@inesctec.pt" target="_blank">hmf@inesctec.pt</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hello,<br>
<br>
I have posted this question in Stackoverflow and did not get an answer. This seems to be a basic usage question and am therefore sending it here.<br>
<br>
I have following code snippet that attempts to do a grid search in which one of the grid parameters are the distance metrics to be used for the KNN algorithm. The example below fails if I use "wminkowski", "seuclidean" or "mahalanobis" distances metrics.<br>
<br>
# Define the parameter values that should be searched<br>
k_range = range(1,31)<br>
weights = ['uniform' , 'distance']<br>
algos = ['auto', 'ball_tree', 'kd_tree', 'brute']<br>
leaf_sizes = range(10, 60, 10)<br>
metrics = ["euclidean", "manhattan", "chebyshev", "minkowski", "mahalanobis"]<br>
<br>
param_grid = dict(n_neighbors = list(k_range), weights = weights, algorithm = algos, leaf_size = list(leaf_sizes), metric=metrics)<br>
param_grid<br>
<br>
# Instantiate the algorithm<br>
knn = KNeighborsClassifier(n_neighbors=10)<br>
<br>
# Instantiate the grid<br>
grid = GridSearchCV(knn, param_grid=param_grid, cv=10, scoring='accuracy', n_jobs=-1)<br>
<br>
# Fit the models using the grid parameters<br>
grid.fit(X,y)<br>
<br>
I assume this is because I have to set or define the ranges for the various distance parameters (for example p, w for “wminkowski” - WMinkowskiDistance ). The "minkowski" distance may be working because its "p" parameter has the default 2.<br>
<br>
So my questions are:<br>
<br>
1. Can we set the range of parameters for the distance metrics for the grid search and if so how?<br>
2. Can we set the value of a parameters for the distance metrics for the grid search and if so how?<br>
<br>
Hope the question is clear.<br>
TIA<br>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote></div><br></div></div></div>
<br>_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
<br></blockquote></div><br></div>