Mailman 3 January 2017 - scikit-learn

Bookmarklet to view documentation on CircleCI
by Joel Nothman April 7, 2017

April 7, 2017

At https://gist.github.com/jnothman/bf76d02f60af6476221ec65c63c77e60 I've created a bookmarklet which, when viewing a pull request page for which the CircleCI build has finished, will identify the circle build number and open a new tab with the changed documentation files corresponding to that PR.

5 8

Preparing a scikit-learn 0.18.2 bugfix release
by Olivier Grisel March 26, 2017

March 26, 2017

Hi all, I think we should release 0.18.2 to get some important fixes and make it easy to release Python 3.6 wheel package for all the operating systems using the automated procedure. I identified a couple of PR to backport to 0.18.X to prepare the 0.18.2 release. Are there any other important recently fixed bugfs people would like to see backported in this release? https://github.com/scikit-learn/scikit-learn/milestone/23?closed=1 Best, -- Olivier http://twitter.com/ogrisel - http://github.com/ogrisel

5 16

GSOC call for mentors
by Andreas Mueller Feb. 21, 2017

Feb. 21, 2017

Hey all. It's that time of the year again. Are we planning on participating in GSOC? If so, we need mentors and projects. It's unlikely that I'll have time to help with either in any substantial way. If we want to participate, I think we should try to be a bit more organized than last year ;) Andy Sent from phone. Please excuse spelling and brevity.

7 9

Random StratifiedKFold Grid Search CV
by Raga Markely Jan. 30, 2017

Jan. 30, 2017

Hello, I was trying to do repeated Grid Search CV (20 repeats). I thought that each time I call GridSearchCV, the training and test sets separated in different splits would be different. However, I got the same best_params_ and best_scores_ for all 20 repeats. It looks like the training and test sets are separated in identical folds in each run? Just to clarify, e.g. I have the following data: 0,1,2,3,4. Class 1 = [0,1,2] and Class 2 = [3,4]. Suppose I call cv = 2. The split is always for instance [0,3] [1,2,4] in each repeat, and I couldn't get [1,3] [0,2,4] or other combinations. If I understand correctly, GridSearchCV uses StratifiedKFold when I enter cv = integer. The StratifiedKFold command has random state; I wonder if there is anyway I can make the the training and test sets randomly separated each time I call the GridSearchCV? Just a note, I used the following classifiers: Logistic Regression, KNN, SVC, Kernel SVC, Random Forest, and had the same observation regardless of the classifiers. Thank you very much! Raga

3 11

K-SVD implementation PR (needs 2nd review)
by Умнов Алексей (Alexey Umnov) Jan. 29, 2017

Jan. 29, 2017

1 0

Scores in Cross Validation
by Raga Markely Jan. 27, 2017

Jan. 27, 2017

Hello, I have 2 questions regarding cross_val_score. 1. Do the scores returned by cross_val_score correspond to only the test set or the whole data set (training and test sets)? I tried to look at the source code, and it looks like it returns the score of only the test set (line 145: "return_train_score=False") - I am not sure if I am reading the codes properly, though.. https://github.com/scikit-learn/scikit-learn/blob/14031f6/sklearn/model_ selection/_validation.py#L36 I came across the paper below and the authors use the score of the whole dataset when the author performs repeated nested loop, grid search cv, etc.. e.g. see algorithm 1 (line 1c) and 2 (line 2d) on page 3. https://jcheminf.springeropen.com/articles/10.1186/1758-2946-6-10 I wonder what's the pros and cons of using the accuracy score of the whole dataset vs just the test set.. any thoughts? 2. On line 283 of the cross_val_score source code, there is a function _score. However, I can't find where this function is called. Could you let me know where this function is called? Thank you very much! Raga

3 5

top N accuracy classification metric
by Johnson, Jeremiah Jan. 26, 2017

Jan. 26, 2017

Hi all, It's common to use a top-n accuracy metric for multi-class classification problems, where for each observation the prediction is the set of probabilities for each of the classes, and a prediction is top-N accurate if the correct class is among the N highest predicted probability classes. I've written a simple implementation, but I don't think it quite fits the sklearn api. Specifically, _check_targets objects to the the continuous-multioutput format of the predictions for a classification task. Is there any interest in including a metric like this? I'd be happy to submit a pull request. Jeremiah

2 2

(personal) Survey for future scikit-learn development
by Andreas Mueller Jan. 26, 2017

Jan. 26, 2017

Hey all. I created a survey to prioritize and justify (to people that give me money) future scikit-learn development. It would be great if you could answer it, it should be pretty sort (it's 10 questions, mostly multiple choice). Feel free to share, more replies are better ;) https://www.surveymonkey.com/r/GJFK32S Disclaimer: While I will share the results with the project (and anyone that cares), I want to make clear that that this is not an "official scikit-learn survey" in that it wasn't designed or endorsed by the whole team. Also, the results are not binding for anyone, though I will use them to steer my work and projects, and I think the rest of the project will certainly take the input into account. Thanks! Andy

1 0

class_weight: How to assign a higher weightage to values in a specific column as opposed to values in another column
by Debabrata Ghosh Jan. 24, 2017

Jan. 24, 2017

Thanks Josh for your quick feedback ! It's quite helpful indeed . Further to it , I am having another burning question. In my sample dataset , I have 2 label columns (let's say x and y) My objective is to give the labels within column 'x' 10 times more weight as compared to labels within column y. My question is the parameter class_weight={0: 1, 1: 10} works for a single column, i.e., within a single column I have assigned 10 times weight to the positive labels. But my objective is to provide a 10 times weight to the positive labels within column 'x' as compared to the positive labels within column 'y'. May I please get a feedback from you around how to achieve this please. Thanks for your help in advance ! On Mon, Jan 23, 2017 at 9:56 AM, Josh Vredevoogd <cleverless(a)gmail.com> wrote: > If you undersample, taking only 10% of the negative class, the classifier > will see different combinations of attributes and produce a different fit > to explain those distributions. In the worse case, imagine you are > classifying birds and through sampling you eliminate all `red` examples. > Your classifier likely now will not understand that red objects can be > birds. That's an overly simple example, but given a classifier capable of > exploring and explaining feature combinations, less obvious versions of > this are bound to happen. > > The extrapolation only works in the other direction: if you manually > duplicate samples by the sampling factor, you should get the exact same fit > as if you increased the class weight. > > Hope that helps, > Josh > > > On Sun, Jan 22, 2017 at 5:00 AM, Debabrata Ghosh <mailfordebu(a)gmail.com> > wrote: > >> Thanks Josh ! >> >> I have used the parameter class_weight={0: 1, 1: 10} and the model code >> has run successfully. However, just to get a further clarity around it's >> concept, I am having another question for you please. I did the following 2 >> tests: >> >> 1. In my dataset , I have 1 million negative classes and 10,000 positive >> classes. First I ran my model code without supplying any class_weight >> parameter and it gave me certain True Positive and False Positive results. >> >> 2. Now in the second test, I had the same 1 million negative classes but >> reduced the positive classes to 1000 . But this time, I supplied the >> parameter class_weight={0: 1, 1: 10} and got my True Positive and False >> Positive Results >> >> My question is , when I multiply the results obtained from my second test >> with a factor of 10, I don't match with the results obtained from my first >> test. In other words, say I get the true positive against a threshold from >> the second test as 8 , while the true positive from the first test against >> the same threshold is 260. I am getting similar observations for the false >> positive results wherein if I multiply the results obtained in the second >> test by 10, I don't come close to the results obtained from the first set. >> >> Is my expectation correct ? Is my way of executing the test (i.e., >> reducing the the positive classes by 10 times and then feeding a class >> weight of 10 times the negative classes) and comparing the results with a >> model run without any class weight parameter correct ? >> >> Please let me know as per your convenience as this will help me a big way >> to understand the concept further. >> >> Thanks in advance ! >> >> On Sun, Jan 22, 2017 at 1:56 AM, Josh Vredevoogd <cleverless(a)gmail.com> >> wrote: >> >>> The class_weight parameter doesn't behave the way you're expecting. >>> >>> The value in class_weight is the weight applied to each sample in that >>> class - in your example, each class zero sample has weight 0.001 and each >>> class one sample has weight 0.999, so each class one samples carries 999 >>> times the weight of a class zero sample. >>> >>> If you would like each class one sample to have ten times the weight, >>> you would set `class_weight={0: 1, 1: 10}` or `class_weight={0:0.1, 1:1}` >>> equivalently. >>> >>> >>> On Sat, Jan 21, 2017 at 10:18 AM, Debabrata Ghosh <mailfordebu(a)gmail.com >>> > wrote: >>> >>>> Hi All, >>>> Greetings ! >>>> >>>> I have a very basic question regarding the usage of the >>>> parameter class_weight in scikit learn's Random Forest Classifier's fit >>>> method. >>>> >>>> I have a fairly unbalanced sample and my positive class : >>>> negative class ratio is 1:100. In other words, I have a million records >>>> corresponding to negative class and 10,000 records corresponding to >>>> positive class. I have trained the random forest classifier model using the >>>> above record set successfully. >>>> >>>> Further, for a different problem, I want to test the >>>> parameter class_weight. So, I am setting the class_weight as [0:0.001 , >>>> 1:0.999] and I have tried running my model on the same dataset as mentioned >>>> in the above paragraph but with the positive class records reduced to 1000 >>>> [because now each positive class is given approximately 10 times more >>>> weight than a negative class]. However, the model run results are very very >>>> different between the 2 runs (with and without class_weight). And I >>>> expected a similar run results. >>>> >>>> Would you please be able to let me know where am I >>>> getting wrong. I know it's something silly but just want to improve on my >>>> concept. >>>> >>>> Thanks ! >>>> >>>> _______________________________________________ >>>> scikit-learn mailing list >>>> scikit-learn(a)python.org >>>> https://mail.python.org/mailman/listinfo/scikit-learn >>>> >>>> >>> >>> _______________________________________________ >>> scikit-learn mailing list >>> scikit-learn(a)python.org >>> https://mail.python.org/mailman/listinfo/scikit-learn >>> >>> >> >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org >> https://mail.python.org/mailman/listinfo/scikit-learn >> >> > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn > >

3 3

Query regarding parameter class_weight in Random Forest Classifier
by Debabrata Ghosh Jan. 23, 2017

Jan. 23, 2017

Hi All, Greetings ! I have a very basic question regarding the usage of the parameter class_weight in scikit learn's Random Forest Classifier's fit method. I have a fairly unbalanced sample and my positive class : negative class ratio is 1:100. In other words, I have a million records corresponding to negative class and 10,000 records corresponding to positive class. I have trained the random forest classifier model using the above record set successfully. Further, for a different problem, I want to test the parameter class_weight. So, I am setting the class_weight as [0:0.001 , 1:0.999] and I have tried running my model on the same dataset as mentioned in the above paragraph but with the positive class records reduced to 1000 [because now each positive class is given approximately 10 times more weight than a negative class]. However, the model run results are very very different between the 2 runs (with and without class_weight). And I expected a similar run results. Would you please be able to let me know where am I getting wrong. I know it's something silly but just want to improve on my concept. Thanks !

2 3