Mailman 3 February 2017 - scikit-learn

Bookmarklet to view documentation on CircleCI
by Joel Nothman April 7, 2017

April 7, 2017

At https://gist.github.com/jnothman/bf76d02f60af6476221ec65c63c77e60 I've created a bookmarklet which, when viewing a pull request page for which the CircleCI build has finished, will identify the circle build number and open a new tab with the changed documentation files corresponding to that PR.

5 8

Preparing a scikit-learn 0.18.2 bugfix release
by Olivier Grisel March 26, 2017

March 26, 2017

Hi all, I think we should release 0.18.2 to get some important fixes and make it easy to release Python 3.6 wheel package for all the operating systems using the automated procedure. I identified a couple of PR to backport to 0.18.X to prepare the 0.18.2 release. Are there any other important recently fixed bugfs people would like to see backported in this release? https://github.com/scikit-learn/scikit-learn/milestone/23?closed=1 Best, -- Olivier http://twitter.com/ogrisel - http://github.com/ogrisel

5 16

Scipy 2017
by Andreas Mueller March 5, 2017

March 5, 2017

Hey folks. Who's coming to scipy this year? Any volunteers for tutorials? I'm happy to be part of it but doing 7h by myself is a bit much ;) Andy

5 5

Women in Machine Learning and Data Science Sprint next Weekend (also call for help)
by Andreas Mueller March 1, 2017

March 1, 2017

Hey all. There's gonna be an introductory scikit-learn sprint at NYC on Saturday that a local Women's DS/ML group is organizing with me. I feel like we could do a bit more to improve (gender) diversity in the scipy/pydata space, and so I think this will be cool. If anyone wants to review code on Saturday that would be a great help for people getting started. Also, if anyone wants to help beforehand, making sure there is enough "easy" and "need contributor" issues tagged is important, as well as ensuring that all the tagged issues actually still need contributors. I'll try to do as much of these as I can but my time is limited these days :( Thanks y'all! Andy

2 4

GSoC, 2017 - Parallel Decision Tree Building
by Aman Dalmia Feb. 28, 2017

Feb. 28, 2017

Hello everyone, I am a pre-final year student studying Electronics & Communication Engineering at IIT Guwahati. I am a member of Prof. Amit Sethi <http://www.iitg.ernet.in/amitsethi/teaching.html>'s research group where I work on cancer recurrence prediction using deep learning and have also started working with Prof. Ashish Anand <http://Prof.%20Ashish%20Anand>, using NLP for genome sequencing. I want to contribute to scikit-learn working on the project 'Parallel Decision Tree Building' for GSoC, 2017. I have been contributing to scikit-learn for the past few weeks working on issues across different modules. Although I am familiar with the tree building algorithms, I have not worked a lot on the tree module of scikit-learn and hence, am I trying to familiarize myself by working on these issues: https://github.com/scikit-learn/scikit-learn/issues/4225 https://github.com/scikit-learn/scikit-learn/issues/6557 Please let me know as to what should be the next steps that I need to follow for building a good proposal. Thank you, Aman Dalmia, Pre-final year student, Electronics & Communication Engineering, IIT Guwahati, +91-8011492025

2 1

Clustering 4 dimensional data
by Rohan Koodli Feb. 28, 2017

Feb. 28, 2017

I'm having trouble understanding how to cluster multidimensional data. Specifically, a 4 dimensional array. test = [[[[3,10],[1,5],[3,18]],[[3,1],[0,0],[0,0]],[[3,3],[1,5],[0,0]]],[[[1,5],[2,7],[0,0]],[[1,7],[0,0],[0,0]],[[0,0],[0,0],[0,0]]]] from sklearn import mixture gmm = mixture.GMM() gmm.fit(test) The code returns the following error: "Found array with dim 4. GMM expected <= 2." Do I need to change the way my data is formatted? Is there a way of doing clustering on 4 dimensional data?

5 4

Re: [scikit-learn] scikit-learn Digest, Vol 11, Issue 33
by Ludovico Coletta Feb. 28, 2017

Feb. 28, 2017

Dear Sebastian, this was exactly what we did but is not working. cv_final[0][0] and cv_final[0][1] hold the training and test indices for the first fold (outer loop), while cv_final_nested[0][0] and cv_final_nested[0][1] hold the indices for the parameter optimization for the first fold (inner loop, training and test respectively). You are probably right, there must be a (I hope so) little error somewhere. I will try again in the next days. Thank you for your time Ludovico ________________________________ Da: scikit-learn <scikit-learn-bounces+ludo25_90=hotmail.com(a)python.org> per conto di scikit-learn-request(a)python.org <scikit-learn-request(a)python.org> Inviato: martedì 28 febbraio 2017 00.57 A: scikit-learn(a)python.org Oggetto: scikit-learn Digest, Vol 11, Issue 33 Send scikit-learn mailing list submissions to scikit-learn(a)python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... or, via email, send a message with subject or body 'help' to scikit-learn-request(a)python.org You can reach the person managing the list at scikit-learn-owner(a)python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of scikit-learn digest..." Today's Topics: 1. Re: Control over the inner loop in GridSearchCV (Sebastian Raschka) 2. R: scikit-learn Digest, Vol 11, Issue 32 (Ludovico Coletta) ---------------------------------------------------------------------- Message: 1 Date: Mon, 27 Feb 2017 17:47:02 -0500 From: Sebastian Raschka <se.raschka(a)gmail.com> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] Control over the inner loop in GridSearchCV Message-ID: <CE5F587A-D08F-4759-AD81-0873E90376F9(a)gmail.com> Content-Type: text/plain; charset=utf-8 Hi, Ludovico, my bet is that there is an issue with the format of the object that you pass to the `cv` param of the GridSearchCV. What you need is e.g., " ? An iterable yielding train, test splits.? Or more specifically, say you have a generator, my_gen, that is yielding these splits, the way the indices whould be organized would be: list(my_gen)[0][0] # stores an array of indices used as training fold in the 1st round # e.g., sth like np.array([ 0, 1, 2, 3, 4, 5, 6, ?]) list(my_gen)[0][1] # stores an array of indices used as test fold in the 1st round # e.g., sth like np.array([ 102, 103, 104, 105, 106, 107, 108, ?]) list(my_gen)[1][0] # stores an array of indices used as training fold in the 2nd round my_gen[1][1] # stores an array of indices used as test fold in the 2nd round list(my_gen)[2][0] # stores an array of indices used as training fold in the 3rd round list(my_gen)[2][1] # stores an array of indices used as test fold in the 3rd round Hope that helps. Best, Sebastian > The following did not work. This is what we get --> ValueError: too many values to unpack > On Feb 27, 2017, at 5:13 PM, Ludovico Coletta <ludo25_90(a)hotmail.com> wrote: > > Dear Sebastian, > > thank you for the quick answer. > > The data is stored in a numpy array (shape: 68, 24). We are using scikit 18.1 > > I saw that I wrote something wrong in previous email. Your solution is indeed correct if we leave Scikit decide how to manage the inner loop. This is what we did at the beginning. By doing so, we noticed that the classifier's perfomance decrease (in comparison to a non-optimised classifier). We would like to control the inner split and we need to store the metrics for each fold > > The way we obtained the indices for the optimization, train and test phase is the equivalent of something like that: > > rs = ShuffleSplit(n_splits=9, test_size=.25,random_state=42) > indices_for_each_cv = list(rs.split(data[0:11])) > > Maybe I can make myself clearer if I write what we would like to achieve for the first cross validation fold (I acknowledge that the previous email was quite a mess, sorry). Outer loop: 48 for training, 20 for testing. Of the 48 training subjects, we would like to use 42 for optimization, 6 for testing the parameters. We got the indices so that we match the different scanners even in the optimization phase, but we are not able to pass them to GridSearch object. > > The following did not work. This is what we get --> ValueError: too many values to unpack > > ii = 0 > > while ii < len(cv_final): > # fit and predict > > clf = GridSearchCV( > pipeline, > param_grid=param_grid, > verbose=1, > cv = cv_final_nested[ii], # how to split the 48 train subjects for the optimization > scoring='roc_auc', > n_jobs= -1) > > clf.fit(data[cv_final[ii][0]], y[cv_final[ii][0]]) # the train data of the outer loop for the first (i.e. the 48 subjects) > predictions.append(clf.predict(data[cv_final[ii][1]])) # Predict the 20 subjects left out for test in the outer loop > > ii = ii + 1 > > This however works and should be (more or less) what we would like to achieve with the above loop. However, extracting the best parameters for each fold in order to predict the left out data seems impossible or very laborious. > > clf = GridSearchCV( > pipeline, > > param_grid=param_grid, > verbose=1, > cv = cv_final_nested, > scoring='roc_auc', > n_jobs= -1) > > clf.fit(data,y) > > > Any hint on how to solve this problem would be really appreciated. > > Best > Ludovico > > > > > Da: scikit-learn <scikit-learn-bounces+ludo25_90=hotmail.com(a)python.org> per conto di scikit-learn-request(a)python.org <scikit-learn-request(a)python.org> > Inviato: luned? 27 febbraio 2017 17.27 > A: scikit-learn(a)python.org > Oggetto: scikit-learn Digest, Vol 11, Issue 29 > > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > scikit-learn Info Page - Python > mail.python.org > To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > > > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. GSoC 2017 (Gael Varoquaux) > 2. Control over the inner loop in GridSearchCV (Ludovico Coletta) > 3. Re: Control over the inner loop in GridSearchCV > (Sebastian Raschka) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 27 Feb 2017 11:58:35 +0100 > From: Gael Varoquaux <gael.varoquaux(a)normalesup.org> > To: Scikit-learn user and developer mailing list > <scikit-learn(a)python.org> > Subject: [scikit-learn] GSoC 2017 > Message-ID: <20170227105835.GC2041043(a)phare.normalesup.org> > Content-Type: text/plain; charset=iso-8859-1 > > Hi, > > Students have been inquiring about the GSoC (Google Summer of Code) with > scikit-learn, and the core team has been quite silent about team. > > I am happy to announce that we will be taking part in the scikit-learn > again. The reason that we decided to do this is to give a chance to the > young, talented, and motivated students. > > Importantly, our most limiting resource is the time of our experienced > developers. This is clearly visible from the number of pending pull > requests. Hence, we need students to be very able and independent. This > of course means that they will be getting supervision from mentors. Such > supervision is crucial for moving forward with a good project, that > delivers mergeable code. However, we will need the students to be very > good at interacting efficiently with the mentors. Also, I should stress > that we will be able to take only a very few numbers of students. > > With that said, let me introduce the 2017 GSoC for scikit-learn. We have > set up a wiki page which summarizes the experiences from last year and > the ideas for this year: > https://github.com/scikit-learn/scikit-learn/wiki/Google-summer-of-code-(GS… Google summer of code (GSOC) 2017 · scikit-learn/scikit-learn Wiki · GitHub<https://github.com/scikit-learn/scikit-learn/wiki/Google-summer-of-code-(GS…> github.com scikit-learn: machine learning in Python > Google summer of code (GSOC) 2017 ? scikit-learn/scikit-learn Wiki ? GitHub > github.com > scikit-learn: machine learning in Python > > > > Interested students should declare their interest on the mailing list, > and discuss with possible mentors here. Factors of success will be > > * careful work on a good proposal, that takes on of the ideas on the wiki > but breaks it down in a realistic plan with multiple steps and shows a > good understanding of the problem. > > * demonstration of the required skillset via successful pull requests in > scikit-learn. > > Cheers, > > Ga?l > > > -- > Gael Varoquaux > Researcher, INRIA Parietal > NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France > Phone: ++ 33-1-69-08-79-68 > http://gael-varoquaux.info http://twitter.com/GaelVaroquaux Gael Varoquaux (@GaelVaroquaux) | Twitter<http://twitter.com/GaelVaroquaux> twitter.com The latest Tweets from Gael Varoquaux (@GaelVaroquaux). Researcher and geek: ►Brain, Data, & Computational science ►#python #pydata #sklearn ►Machine learning for fMRI ►Photography on @artgael. Paris, France Gaël Varoquaux: computer / data / brain science<http://gael-varoquaux.info/> gael-varoquaux.info Gaël Varoquaux, computer / data / brain science ... Latest posts . misc personnal programming science Our research in 2016: personal scientific highlights > Gael Varoquaux (@GaelVaroquaux) | Twitter > twitter.com > The latest Tweets from Gael Varoquaux (@GaelVaroquaux). Researcher and geek: ?Brain, Data, & Computational science ?#python #pydata #sklearn ?Machine learning for fMRI ?Photography on @artgael. Paris, France > > Ga?l Varoquaux: computer / data / brain science > gael-varoquaux.info > Ga?l Varoquaux, computer / data / brain science ... Latest posts . misc personnal programming science Our research in 2016: personal scientific highlights > > > > > ------------------------------ > > Message: 2 > Date: Mon, 27 Feb 2017 14:27:59 +0000 > From: Ludovico Coletta <ludo25_90(a)hotmail.com> > To: "scikit-learn(a)python.org" <scikit-learn(a)python.org> > Subject: [scikit-learn] Control over the inner loop in GridSearchCV > Message-ID: > <BLUPR0301MB2017606E3E103266BBAB5E698C570(a)BLUPR0301MB2017.namprd03.prod.outlook.com> > > Content-Type: text/plain; charset="iso-8859-1" > > Dear Scikit experts, > > > we am stucked with GridSearchCV. Nobody else was able/wanted to help us, we hope you will. > > > We are analysing neuroimaging data coming from 3 different MRI scanners, where for each scanner we have a healthy group and a disease group. We would like to merge the data from the 3 different scanners in order to classify the healthy subjects from the one who have the disease. > > > The problem is that we can almost perfectly classify the subjects according to the scanner (e.g. the healthy subjects from scanner 1 and scanner 2). We are using a custom cross validation schema to account for the different scanners: when no hyper-parameter (SVM) optimization is performed, everything is straightforward. Problems arise when we would like to perform hyperparameter optimization: in this case we need to balance for the different scanner in the optimization phase as well. We also found a custom cv schema for this, but we are not able to pass it to GridSearchCV object. We would like to get something like the following: > > > pipeline = Pipeline([('scl', StandardScaler()), > ('sel', RFE(estimator,step=0.2)), > ('clf', SVC(probability=True, random_state=42))]) > > > param_grid = [{'sel__n_features_to_select':[22,15,10,2], > 'clf__C': np.logspace(-3, 5, 100), > 'clf__kernel':['linear']}] > > clf = GridSearchCV(pipeline, > param_grid=param_grid, > verbose=1, > scoring='roc_auc', > n_jobs= -1) > > # cv_final is the custom cv for the outer loop (9 folds) > > ii = 0 > > while ii < len(cv_final): > # fit and predict > > clf.fit(data[?]], y[[?]]) > predictions.append(clf.predict(data[cv_final[ii][1]])) # outer test data > ii = ii + 1 > > > We tried almost everything. When we define clf in the loop, we pass the -ith cv_nested as cv argument, and we fit it on the training data of the -ith custom_cv fold, we get an "Too many values to unpack" error. On the other end, when we try to pass the nested -ith cv fold as cv argument for clf, and we call fit on the same cv_nested fold, we get an "Index out of bound" error. > > Two questions: > > 1) Is there any workaround to avoid the split when clf is called without a cv argument? > > 2) We suppose that for hyperparameter optimization the test data is removed from the dataset and a new dataset is created. Is this true? In this case we only have to adjust the indices accordingly > > > Thank your for your time and sorry for the long text > > Ludovico > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20170227/e80777cb…> > > ------------------------------ > > Message: 3 > Date: Mon, 27 Feb 2017 11:27:24 -0500 > From: Sebastian Raschka <se.raschka(a)gmail.com> > To: Scikit-learn user and developer mailing list > <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] Control over the inner loop in > GridSearchCV > Message-ID: <FC403FD1-9A00-424A-8453-9D60FE176C92(a)gmail.com> > Content-Type: text/plain; charset=utf-8 > > Hi, Ludovico, > what format (shape) is data in? Are these the arrays from a Kfold iterator? In this case, the ?question marks? in your code snippet should simply be the train and validation subset indices generated by the KFold generator. E.g., > > skfold = StratifiedKFold(y=y_train, n_folds=5, shuffle=True, random_state=1) > for outer_train_idx, outer_valid_idx in skfold: > ? > gridsearch_object.fit(X_train[outer_train_idx], y_train[outer_train_idx]) > > > > > On the other end, when we try to pass the nested -ith cv fold as cv argument for clf, and we call fit on the same cv_nested fold, we get an "Index out of bound" error. > > Two questions: > > Are you using an version older than scikit-learn 0.18? Techically, the GridSearchCV, RandomizedSearchCV, cross_val_score ? should all support iterables that of train_ and test_indices e.g.: > > outer_cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=1) > > for name, gs_est in sorted(gridcvs.items()): > nested_score = cross_val_score(gs_est, > X=X_train, > y=y_train, > cv=outer_cv, > n_jobs=1) > > > Best, > Sebastian > > > On Feb 27, 2017, at 9:27 AM, Ludovico Coletta <ludo25_90(a)hotmail.com> wrote: > > > > Dear Scikit experts, > > > > we am stucked with GridSearchCV. Nobody else was able/wanted to help us, we hope you will. > > > > We are analysing neuroimaging data coming from 3 different MRI scanners, where for each scanner we have a healthy group and a disease group. We would like to merge the data from the 3 different scanners in order to classify the healthy subjects from the one who have the disease. > > > > The problem is that we can almost perfectly classify the subjects according to the scanner (e.g. the healthy subjects from scanner 1 and scanner 2). We are using a custom cross validation schema to account for the different scanners: when no hyper-parameter (SVM) optimization is performed, everything is straightforward. Problems arise when we would like to perform hyperparameter optimization: in this case we need to balance for the different scanner in the optimization phase as well. We also found a custom cv schema for this, but we are not able to pass it to GridSearchCV object. We would like to get something like the following: > > > > pipeline = Pipeline([('scl', StandardScaler()), > > ('sel', RFE(estimator,step=0.2)), > > ('clf', SVC(probability=True, random_state=42))]) > > > > > > param_grid = [{'sel__n_features_to_select':[22,15,10,2], > > 'clf__C': np.logspace(-3, 5, 100), > > 'clf__kernel':['linear']}] > > > > clf = GridSearchCV(pipeline, > > param_grid=param_grid, > > verbose=1, > > scoring='roc_auc', > > n_jobs= -1) > > > > # cv_final is the custom cv for the outer loop (9 folds) > > > > ii = 0 > > > > while ii < len(cv_final): > > # fit and predict > > > > clf.fit(data[?]], y[[?]]) > > predictions.append(clf.predict(data[cv_final[ii][1]])) # outer test data > > ii = ii + 1 > > > > We tried almost everything. When we define clf in the loop, we pass the -ith cv_nested as cv argument, and we fit it on the training data of the -ith custom_cv fold, we get an "Too many values to unpack" error. On the other end, when we try to pass the nested -ith cv fold as cv argument for clf, and we call fit on the same cv_nested fold, we get an "Index out of bound" error. > > Two questions: > > 1) Is there any workaround to avoid the split when clf is called without a cv argument? > > 2) We suppose that for hyperparameter optimization the test data is removed from the dataset and a new dataset is created. Is this true? In this case we only have to adjust the indices accordingly > > > > Thank your for your time and sorry for the long text > > Ludovico > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > scikit-learn Info Page - Python > mail.python.org > To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > > > > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > scikit-learn Info Page - Python > mail.python.org > To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > > > > > ------------------------------ > > End of scikit-learn Digest, Vol 11, Issue 29 > ******************************************** > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... ------------------------------ Message: 2 Date: Mon, 27 Feb 2017 23:56:59 +0000 From: Ludovico Coletta <ludo25_90(a)hotmail.com> To: "scikit-learn(a)python.org" <scikit-learn(a)python.org> Subject: [scikit-learn] R: scikit-learn Digest, Vol 11, Issue 32 Message-ID: <BLUPR0301MB20178F820111B57E9C4CF95F8C570(a)BLUPR0301MB2017.namprd03.prod.outlook.com> Content-Type: text/plain; charset="us-ascii" Dear Gael, This will probably be the case here, but we would like to exclude the scanner-factor from the possible explanations. We are still lucky that we are not in situation where the number of features >> number of samples. Best Ludovico -------- Messaggio originale -------- Da: scikit-learn-request(a)python.org Data: 27/02/17 23:49 (GMT+01:00) A: scikit-learn(a)python.org Oggetto: scikit-learn Digest, Vol 11, Issue 32 Send scikit-learn mailing list submissions to scikit-learn(a)python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... or, via email, send a message with subject or body 'help' to scikit-learn-request(a)python.org You can reach the person managing the list at scikit-learn-owner(a)python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of scikit-learn digest..." Today's Topics: 1. Re: scikit-learn Digest, Vol 11, Issue 29 (Gael Varoquaux) 2. Re: GSoC 2017 (Joel Nothman) 3. Re: GSoC 2017 (Pradeep Thalasta) ---------------------------------------------------------------------- Message: 1 Date: Mon, 27 Feb 2017 23:19:33 +0100 From: Gael Varoquaux <gael.varoquaux(a)normalesup.org> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] scikit-learn Digest, Vol 11, Issue 29 Message-ID: <20170227221933.GC2369856(a)phare.normalesup.org> Content-Type: text/plain; charset=iso-8859-1 On Mon, Feb 27, 2017 at 10:13:04PM +0000, Ludovico Coletta wrote: > The data is stored in a numpy array (shape: 68, 24). We are using scikit 18.1 > I saw that I wrote something wrong in previous email. Your solution is indeed > correct if we leave Scikit decide how to manage the inner loop. This is what we > did at the beginning. By doing so, we noticed that the classifier's perfomance > decrease (in comparison to a non-optimised classifier). With 68 samples, it is not that surprising the model-selection with cross-validation is not able to select a good model. We found the same problem in brain imaging data [1], and it's an intrinsic problem due to small sample sizes: cross-validation is just not very accurate in these settings. Ga?l [1] https://arxiv.org/abs/1606.05201 [1606.05201] Assessing and tuning brain decoders: cross ...<https://arxiv.org/abs/1606.05201> arxiv.org Submission history From: Gael Varoquaux Thu, 16 Jun 2016 14:29:28 GMT (785kb,D) [v2] Mon, 7 Nov 2016 15:40:46 GMT (692kb,D) ------------------------------ Message: 2 Date: Tue, 28 Feb 2017 09:34:43 +1100 From: Joel Nothman <joel.nothman(a)gmail.com> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] GSoC 2017 Message-ID: <CAAkaFLXVGAbBjYE2yA+8egUgRvoYnbOgO9QfYTeG0iF9bWUX+A(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hi Pradeep, we would usually only accept candidates who have shown their proficiency and understanding of our package and processes by making some contributions prior to this stage. you are certainly welcome to aim for GSoC 2018 by beginning to develop your familiarity and rapport now. cheers, Joel On 28 Feb 2017 7:01 am, "Pradeep Thalasta" <thalasta(a)usc.edu> wrote: > Hi, > I'm new to open source contribution. Can i take part in GSoc as well? > > > On Mon, Feb 27, 2017 at 11:29 AM, Raghav R V <ragvrv(a)gmail.com> wrote: > >> Or simply a public gist and importantly the link mailed here would do I >> think... >> >> On 27 Feb 2017 8:28 p.m., "Raghav R V" <ragvrv(a)gmail.com> wrote: >> >>> They can still edit a wiki page from their fork of scikit learn I think. >>> So I'd suggest doing that and mailing to this thread, the link to their >>> proposal... >>> >>> On 27 Feb 2017 6:55 p.m., "Nelson Liu" <nfliu(a)uw.edu> wrote: >>> >>>> In past years students made a page on the wiki with their proposal; >>>> this isn't possible anymore due to GitHub permissions. Perhaps an >>>> alternative method for getting feedback should be suggested on the >>>> introduction page? >>>> >>>> Nelson Liu >>>> >>>> On Mon, Feb 27, 2017 at 2:58 AM, Gael Varoquaux < >>>> gael.varoquaux(a)normalesup.org> wrote: >>>> >>>>> Hi, >>>>> >>>>> Students have been inquiring about the GSoC (Google Summer of Code) >>>>> with >>>>> scikit-learn, and the core team has been quite silent about team. >>>>> >>>>> I am happy to announce that we will be taking part in the scikit-learn >>>>> again. The reason that we decided to do this is to give a chance to the >>>>> young, talented, and motivated students. >>>>> >>>>> Importantly, our most limiting resource is the time of our experienced >>>>> developers. This is clearly visible from the number of pending pull >>>>> requests. Hence, we need students to be very able and independent. This >>>>> of course means that they will be getting supervision from mentors. >>>>> Such >>>>> supervision is crucial for moving forward with a good project, that >>>>> delivers mergeable code. However, we will need the students to be very >>>>> good at interacting efficiently with the mentors. Also, I should stress >>>>> that we will be able to take only a very few numbers of students. >>>>> >>>>> With that said, let me introduce the 2017 GSoC for scikit-learn. We >>>>> have >>>>> set up a wiki page which summarizes the experiences from last year and >>>>> the ideas for this year: >>>>> https://github.com/scikit-learn/scikit-learn/wiki/Google-sum Home · scikit-learn/scikit-learn Wiki · GitHub<https://github.com/scikit-learn/scikit-learn/wiki/Google-sum> github.com scikit-learn: machine learning in Python >>>>> mer-of-code-(GSOC)-2017 >>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_scikit-2Dle…> Google summer of code (GSOC) 2017 · scikit-learn/scikit-learn Wiki · GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_scikit-2Dle…> urldefense.proofpoint.com scikit-learn: machine learning in Python >>>>> >>>>> Interested students should declare their interest on the mailing list, >>>>> and discuss with possible mentors here. Factors of success will be >>>>> >>>>> * careful work on a good proposal, that takes on of the ideas on the >>>>> wiki >>>>> but breaks it down in a realistic plan with multiple steps and shows >>>>> a >>>>> good understanding of the problem. >>>>> >>>>> * demonstration of the required skillset via successful pull requests >>>>> in >>>>> scikit-learn. >>>>> >>>>> Cheers, >>>>> >>>>> Ga?l >>>>> >>>>> >>>>> -- >>>>> Gael Varoquaux >>>>> Researcher, INRIA Parietal >>>>> NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France >>>>> Phone: ++ 33-1-69-08-79-68 >>>>> http://gael-varoquaux.info Gaël Varoquaux: computer / data / brain science<http://gael-varoquaux.info/> gael-varoquaux.info Gaël Varoquaux, computer / data / brain science ... Latest posts . misc personnal programming science Our research in 2016: personal scientific highlights >>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__gael-2Dvaroquaux.info&d…> >>>>> http://twitter.com/GaelVaroquaux Gael Varoquaux (@GaelVaroquaux) | Twitter<http://twitter.com/GaelVaroquaux> twitter.com The latest Tweets from Gael Varoquaux (@GaelVaroquaux). Researcher and geek: ►Brain, Data, & Computational science ►#python #pydata #sklearn ►Machine learning for fMRI ►Photography on @artgael. Paris, France >>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_GaelVaroqua…> >>>>> _______________________________________________ >>>>> scikit-learn mailing list >>>>> scikit-learn(a)python.org >>>>> https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... >>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailma…> >>>>> >>>> >>>> >>>> _______________________________________________ >>>> scikit-learn mailing list >>>> scikit-learn(a)python.org >>>> https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailma…> >>>> >>>> >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org >> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.py >> thon.org_mailman_listinfo_scikit-2Dlearn&d=DwICAg&c=clK7kQUT >> WtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=8wN-jbuYw7VyipS2uLHiQg >> &m=WOCvB_ncbkX6zknItZ8JGw5QvsCBNqh2DCc_AxGKj10&s=2HaUcj6htbn >> tv3V5UTTAgAtZk6luVMnqXA9vEOlfJ_k&e= >> >> > > > -- > Regards, > Pradeep Thalasta > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20170228/220a6d01…> ------------------------------ Message: 3 Date: Mon, 27 Feb 2017 14:46:36 -0800 From: Pradeep Thalasta <thalasta(a)usc.edu> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] GSoC 2017 Message-ID: <CAEBU=NM-dqum=1AeKd1+gUG+upZ+Z4+jJ-sMHk+Z1doVHkR6JQ(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Thanks Joel, will start with the contribution soon. On 27 Feb 2017 2:35 pm, "Joel Nothman" <joel.nothman(a)gmail.com> wrote: Hi Pradeep, we would usually only accept candidates who have shown their proficiency and understanding of our package and processes by making some contributions prior to this stage. you are certainly welcome to aim for GSoC 2018 by beginning to develop your familiarity and rapport now. cheers, Joel On 28 Feb 2017 7:01 am, "Pradeep Thalasta" <thalasta(a)usc.edu> wrote: > Hi, > I'm new to open source contribution. Can i take part in GSoc as well? > > > On Mon, Feb 27, 2017 at 11:29 AM, Raghav R V <ragvrv(a)gmail.com> wrote: > >> Or simply a public gist and importantly the link mailed here would do I >> think... >> >> On 27 Feb 2017 8:28 p.m., "Raghav R V" <ragvrv(a)gmail.com> wrote: >> >>> They can still edit a wiki page from their fork of scikit learn I think. >>> So I'd suggest doing that and mailing to this thread, the link to their >>> proposal... >>> >>> On 27 Feb 2017 6:55 p.m., "Nelson Liu" <nfliu(a)uw.edu> wrote: >>> >>>> In past years students made a page on the wiki with their proposal; >>>> this isn't possible anymore due to GitHub permissions. Perhaps an >>>> alternative method for getting feedback should be suggested on the >>>> introduction page? >>>> >>>> Nelson Liu >>>> >>>> On Mon, Feb 27, 2017 at 2:58 AM, Gael Varoquaux < >>>> gael.varoquaux(a)normalesup.org> wrote: >>>> >>>>> Hi, >>>>> >>>>> Students have been inquiring about the GSoC (Google Summer of Code) >>>>> with >>>>> scikit-learn, and the core team has been quite silent about team. >>>>> >>>>> I am happy to announce that we will be taking part in the scikit-learn >>>>> again. The reason that we decided to do this is to give a chance to the >>>>> young, talented, and motivated students. >>>>> >>>>> Importantly, our most limiting resource is the time of our experienced >>>>> developers. This is clearly visible from the number of pending pull >>>>> requests. Hence, we need students to be very able and independent. This >>>>> of course means that they will be getting supervision from mentors. >>>>> Such >>>>> supervision is crucial for moving forward with a good project, that >>>>> delivers mergeable code. However, we will need the students to be very >>>>> good at interacting efficiently with the mentors. Also, I should stress >>>>> that we will be able to take only a very few numbers of students. >>>>> >>>>> With that said, let me introduce the 2017 GSoC for scikit-learn. We >>>>> have >>>>> set up a wiki page which summarizes the experiences from last year and >>>>> the ideas for this year: >>>>> https://github.com/scikit-learn/scikit-learn/wiki/Google-sum Home · scikit-learn/scikit-learn Wiki · GitHub<https://github.com/scikit-learn/scikit-learn/wiki/Google-sum> github.com scikit-learn: machine learning in Python >>>>> mer-of-code-(GSOC)-2017 >>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_scikit-2Dle…> Google summer of code (GSOC) 2017 · scikit-learn/scikit-learn Wiki · GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_scikit-2Dle…> urldefense.proofpoint.com scikit-learn: machine learning in Python >>>>> >>>>> Interested students should declare their interest on the mailing list, >>>>> and discuss with possible mentors here. Factors of success will be >>>>> >>>>> * careful work on a good proposal, that takes on of the ideas on the >>>>> wiki >>>>> but breaks it down in a realistic plan with multiple steps and shows >>>>> a >>>>> good understanding of the problem. >>>>> >>>>> * demonstration of the required skillset via successful pull requests >>>>> in >>>>> scikit-learn. >>>>> >>>>> Cheers, >>>>> >>>>> Ga?l >>>>> >>>>> >>>>> -- >>>>> Gael Varoquaux >>>>> Researcher, INRIA Parietal >>>>> NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France >>>>> Phone: ++ 33-1-69-08-79-68 >>>>> http://gael-varoquaux.info Gaël Varoquaux: computer / data / brain science<http://gael-varoquaux.info/> gael-varoquaux.info Gaël Varoquaux, computer / data / brain science ... Latest posts . misc personnal programming science Our research in 2016: personal scientific highlights >>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__gael-2Dvaroquaux.info&d…> >>>>> http://twitter.com/GaelVaroquaux >>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_GaelVaroqua…> >>>>> _______________________________________________ >>>>> scikit-learn mailing list >>>>> scikit-learn(a)python.org >>>>> https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... >>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailma…> >>>>> >>>> >>>> >>>> _______________________________________________ >>>> scikit-learn mailing list >>>> scikit-learn(a)python.org >>>> https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailma…> >>>> >>>> >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org >> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.py >> thon.org_mailman_listinfo_scikit-2Dlearn&d=DwICAg&c=clK7kQUT >> WtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=8wN-jbuYw7VyipS2uLHiQg >> &m=WOCvB_ncbkX6zknItZ8JGw5QvsCBNqh2DCc_AxGKj10&s=2HaUcj6htbn >> tv3V5UTTAgAtZk6luVMnqXA9vEOlfJ_k&e= >> >> > > > -- > Regards, > Pradeep Thalasta > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... > <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailma…> > > _______________________________________________ scikit-learn mailing list scikit-learn(a)python.org https://urldefense.proofpoint.com/v2/url?u=https-3A__mail. python.org_mailman_listinfo_scikit-2Dlearn&d=DwICAg&c= clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=8wN-jbuYw7VyipS2uLHiQg&m= SdunRabzBJNpIzcL2t6OsNtuhzVKbzr-DI484v3czLY&s=UEKHpAVTnVYDcV77QhqeRIesWqvTDM Xx9RJj0JlqrKk&e= -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20170227/993b2077…> ------------------------------ Subject: Digest Footer _______________________________________________ scikit-learn mailing list scikit-learn(a)python.org https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... ------------------------------ End of scikit-learn Digest, Vol 11, Issue 32 ******************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20170227/c2181d4b…> ------------------------------ Subject: Digest Footer _______________________________________________ scikit-learn mailing list scikit-learn(a)python.org https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... ------------------------------ End of scikit-learn Digest, Vol 11, Issue 33 ********************************************

1 0

R: scikit-learn Digest, Vol 11, Issue 32
by Ludovico Coletta Feb. 27, 2017

Feb. 27, 2017

Dear Gael, This will probably be the case here, but we would like to exclude the scanner-factor from the possible explanations. We are still lucky that we are not in situation where the number of features >> number of samples. Best Ludovico -------- Messaggio originale -------- Da: scikit-learn-request(a)python.org Data: 27/02/17 23:49 (GMT+01:00) A: scikit-learn(a)python.org Oggetto: scikit-learn Digest, Vol 11, Issue 32 Send scikit-learn mailing list submissions to scikit-learn(a)python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scikit-learn or, via email, send a message with subject or body 'help' to scikit-learn-request(a)python.org You can reach the person managing the list at scikit-learn-owner(a)python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of scikit-learn digest..." Today's Topics: 1. Re: scikit-learn Digest, Vol 11, Issue 29 (Gael Varoquaux) 2. Re: GSoC 2017 (Joel Nothman) 3. Re: GSoC 2017 (Pradeep Thalasta) ---------------------------------------------------------------------- Message: 1 Date: Mon, 27 Feb 2017 23:19:33 +0100 From: Gael Varoquaux <gael.varoquaux(a)normalesup.org> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] scikit-learn Digest, Vol 11, Issue 29 Message-ID: <20170227221933.GC2369856(a)phare.normalesup.org> Content-Type: text/plain; charset=iso-8859-1 On Mon, Feb 27, 2017 at 10:13:04PM +0000, Ludovico Coletta wrote: > The data is stored in a numpy array (shape: 68, 24). We are using scikit 18.1 > I saw that I wrote something wrong in previous email. Your solution is indeed > correct if we leave Scikit decide how to manage the inner loop. This is what we > did at the beginning. By doing so, we noticed that the classifier's perfomance > decrease (in comparison to a non-optimised classifier). With 68 samples, it is not that surprising the model-selection with cross-validation is not able to select a good model. We found the same problem in brain imaging data [1], and it's an intrinsic problem due to small sample sizes: cross-validation is just not very accurate in these settings. Ga?l [1] https://arxiv.org/abs/1606.05201 ------------------------------ Message: 2 Date: Tue, 28 Feb 2017 09:34:43 +1100 From: Joel Nothman <joel.nothman(a)gmail.com> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] GSoC 2017 Message-ID: <CAAkaFLXVGAbBjYE2yA+8egUgRvoYnbOgO9QfYTeG0iF9bWUX+A(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hi Pradeep, we would usually only accept candidates who have shown their proficiency and understanding of our package and processes by making some contributions prior to this stage. you are certainly welcome to aim for GSoC 2018 by beginning to develop your familiarity and rapport now. cheers, Joel On 28 Feb 2017 7:01 am, "Pradeep Thalasta" <thalasta(a)usc.edu> wrote: > Hi, > I'm new to open source contribution. Can i take part in GSoc as well? > > > On Mon, Feb 27, 2017 at 11:29 AM, Raghav R V <ragvrv(a)gmail.com> wrote: > >> Or simply a public gist and importantly the link mailed here would do I >> think... >> >> On 27 Feb 2017 8:28 p.m., "Raghav R V" <ragvrv(a)gmail.com> wrote: >> >>> They can still edit a wiki page from their fork of scikit learn I think. >>> So I'd suggest doing that and mailing to this thread, the link to their >>> proposal... >>> >>> On 27 Feb 2017 6:55 p.m., "Nelson Liu" <nfliu(a)uw.edu> wrote: >>> >>>> In past years students made a page on the wiki with their proposal; >>>> this isn't possible anymore due to GitHub permissions. Perhaps an >>>> alternative method for getting feedback should be suggested on the >>>> introduction page? >>>> >>>> Nelson Liu >>>> >>>> On Mon, Feb 27, 2017 at 2:58 AM, Gael Varoquaux < >>>> gael.varoquaux(a)normalesup.org> wrote: >>>> >>>>> Hi, >>>>> >>>>> Students have been inquiring about the GSoC (Google Summer of Code) >>>>> with >>>>> scikit-learn, and the core team has been quite silent about team. >>>>> >>>>> I am happy to announce that we will be taking part in the scikit-learn >>>>> again. The reason that we decided to do this is to give a chance to the >>>>> young, talented, and motivated students. >>>>> >>>>> Importantly, our most limiting resource is the time of our experienced >>>>> developers. This is clearly visible from the number of pending pull >>>>> requests. Hence, we need students to be very able and independent. This >>>>> of course means that they will be getting supervision from mentors. >>>>> Such >>>>> supervision is crucial for moving forward with a good project, that >>>>> delivers mergeable code. However, we will need the students to be very >>>>> good at interacting efficiently with the mentors. Also, I should stress >>>>> that we will be able to take only a very few numbers of students. >>>>> >>>>> With that said, let me introduce the 2017 GSoC for scikit-learn. We >>>>> have >>>>> set up a wiki page which summarizes the experiences from last year and >>>>> the ideas for this year: >>>>> https://github.com/scikit-learn/scikit-learn/wiki/Google-sum >>>>> mer-of-code-(GSOC)-2017 >>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_scikit-2Dle…> >>>>> >>>>> Interested students should declare their interest on the mailing list, >>>>> and discuss with possible mentors here. Factors of success will be >>>>> >>>>> * careful work on a good proposal, that takes on of the ideas on the >>>>> wiki >>>>> but breaks it down in a realistic plan with multiple steps and shows >>>>> a >>>>> good understanding of the problem. >>>>> >>>>> * demonstration of the required skillset via successful pull requests >>>>> in >>>>> scikit-learn. >>>>> >>>>> Cheers, >>>>> >>>>> Ga?l >>>>> >>>>> >>>>> -- >>>>> Gael Varoquaux >>>>> Researcher, INRIA Parietal >>>>> NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France >>>>> Phone: ++ 33-1-69-08-79-68 >>>>> http://gael-varoquaux.info >>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__gael-2Dvaroquaux.info&d…> >>>>> http://twitter.com/GaelVaroquaux >>>>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__twitter.com_GaelVaroqua…> >>>>> _______________________________________________ >>>>> scikit-learn mailing list >>>>> scikit-learn(a)python.org >>>>> https://mail.python.org/mailman/listinfo/scikit-learn >>>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailma…> >>>>> >>>> >>>> >>>> _______________________________________________ >>>> scikit-learn mailing list >>>> scikit-learn(a)python.org >>>> https://mail.python.org/mailman/listinfo/scikit-learn >>>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.python.org_mailma…> >>>> >>>> >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org >> https://urldefense.proofpoint.com/v2/url?u=https-3A__mail.py >> thon.org_mailman_listinfo_scikit-2Dlearn&d=DwICAg&c=clK7kQUT >> WtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=8wN-jbuYw7VyipS2uLHiQg >> &m=WOCvB_ncbkX6zknItZ8JGw5QvsCBNqh2DCc_AxGKj10&s=2HaUcj6htbn >> tv3V5UTTAgAtZk6luVMnqXA9vEOlfJ_k&e= >> >> > > > -- > Regards, > Pradeep Thalasta > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn > >

1 0

Re: [scikit-learn] scikit-learn Digest, Vol 11, Issue 29
by Ludovico Coletta Feb. 27, 2017

Feb. 27, 2017

Dear Sebastian, thank you for the quick answer. The data is stored in a numpy array (shape: 68, 24). We are using scikit 18.1 I saw that I wrote something wrong in previous email. Your solution is indeed correct if we leave Scikit decide how to manage the inner loop. This is what we did at the beginning. By doing so, we noticed that the classifier's perfomance decrease (in comparison to a non-optimised classifier). We would like to control the inner split and we need to store the metrics for each fold The way we obtained the indices for the optimization, train and test phase is the equivalent of something like that: rs = ShuffleSplit(n_splits=9, test_size=.25,random_state=42) indices_for_each_cv = list(rs.split(data[0:11])) Maybe I can make myself clearer if I write what we would like to achieve for the first cross validation fold (I acknowledge that the previous email was quite a mess, sorry). Outer loop: 48 for training, 20 for testing. Of the 48 training subjects, we would like to use 42 for optimization, 6 for testing the parameters. We got the indices so that we match the different scanners even in the optimization phase, but we are not able to pass them to GridSearch object. The following did not work. This is what we get --> ValueError: too many values to unpack ii = 0 while ii < len(cv_final): # fit and predict clf = GridSearchCV( pipeline, param_grid=param_grid, verbose=1, cv = cv_final_nested[ii], # how to split the 48 train subjects for the optimization scoring='roc_auc', n_jobs= -1) clf.fit(data[cv_final[ii][0]], y[cv_final[ii][0]]) # the train data of the outer loop for the first (i.e. the 48 subjects) predictions.append(clf.predict(data[cv_final[ii][1]])) # Predict the 20 subjects left out for test in the outer loop ii = ii + 1 This however works and should be (more or less) what we would like to achieve with the above loop. However, extracting the best parameters for each fold in order to predict the left out data seems impossible or very laborious. clf = GridSearchCV( pipeline, param_grid=param_grid, verbose=1, cv = cv_final_nested, scoring='roc_auc', n_jobs= -1) clf.fit(data,y) Any hint on how to solve this problem would be really appreciated. Best Ludovico ________________________________ Da: scikit-learn <scikit-learn-bounces+ludo25_90=hotmail.com(a)python.org> per conto di scikit-learn-request(a)python.org <scikit-learn-request(a)python.org> Inviato: lunedì 27 febbraio 2017 17.27 A: scikit-learn(a)python.org Oggetto: scikit-learn Digest, Vol 11, Issue 29 Send scikit-learn mailing list submissions to scikit-learn(a)python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... or, via email, send a message with subject or body 'help' to scikit-learn-request(a)python.org You can reach the person managing the list at scikit-learn-owner(a)python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of scikit-learn digest..." Today's Topics: 1. GSoC 2017 (Gael Varoquaux) 2. Control over the inner loop in GridSearchCV (Ludovico Coletta) 3. Re: Control over the inner loop in GridSearchCV (Sebastian Raschka) ---------------------------------------------------------------------- Message: 1 Date: Mon, 27 Feb 2017 11:58:35 +0100 From: Gael Varoquaux <gael.varoquaux(a)normalesup.org> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: [scikit-learn] GSoC 2017 Message-ID: <20170227105835.GC2041043(a)phare.normalesup.org> Content-Type: text/plain; charset=iso-8859-1 Hi, Students have been inquiring about the GSoC (Google Summer of Code) with scikit-learn, and the core team has been quite silent about team. I am happy to announce that we will be taking part in the scikit-learn again. The reason that we decided to do this is to give a chance to the young, talented, and motivated students. Importantly, our most limiting resource is the time of our experienced developers. This is clearly visible from the number of pending pull requests. Hence, we need students to be very able and independent. This of course means that they will be getting supervision from mentors. Such supervision is crucial for moving forward with a good project, that delivers mergeable code. However, we will need the students to be very good at interacting efficiently with the mentors. Also, I should stress that we will be able to take only a very few numbers of students. With that said, let me introduce the 2017 GSoC for scikit-learn. We have set up a wiki page which summarizes the experiences from last year and the ideas for this year: https://github.com/scikit-learn/scikit-learn/wiki/Google-summer-of-code-(GS… Google summer of code (GSOC) 2017 · scikit-learn/scikit-learn Wiki · GitHub<https://github.com/scikit-learn/scikit-learn/wiki/Google-summer-of-code-(GS…> github.com scikit-learn: machine learning in Python Interested students should declare their interest on the mailing list, and discuss with possible mentors here. Factors of success will be * careful work on a good proposal, that takes on of the ideas on the wiki but breaks it down in a realistic plan with multiple steps and shows a good understanding of the problem. * demonstration of the required skillset via successful pull requests in scikit-learn. Cheers, Ga?l -- Gael Varoquaux Researcher, INRIA Parietal NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France Phone: ++ 33-1-69-08-79-68 http://gael-varoquaux.info http://twitter.com/GaelVaroquaux Gael Varoquaux (@GaelVaroquaux) | Twitter<http://twitter.com/GaelVaroquaux> twitter.com The latest Tweets from Gael Varoquaux (@GaelVaroquaux). Researcher and geek: ►Brain, Data, & Computational science ►#python #pydata #sklearn ►Machine learning for fMRI ►Photography on @artgael. Paris, France Gaël Varoquaux: computer / data / brain science<http://gael-varoquaux.info/> gael-varoquaux.info Gaël Varoquaux, computer / data / brain science ... Latest posts . misc personnal programming science Our research in 2016: personal scientific highlights ------------------------------ Message: 2 Date: Mon, 27 Feb 2017 14:27:59 +0000 From: Ludovico Coletta <ludo25_90(a)hotmail.com> To: "scikit-learn(a)python.org" <scikit-learn(a)python.org> Subject: [scikit-learn] Control over the inner loop in GridSearchCV Message-ID: <BLUPR0301MB2017606E3E103266BBAB5E698C570(a)BLUPR0301MB2017.namprd03.prod.outlook.com> Content-Type: text/plain; charset="iso-8859-1" Dear Scikit experts, we am stucked with GridSearchCV. Nobody else was able/wanted to help us, we hope you will. We are analysing neuroimaging data coming from 3 different MRI scanners, where for each scanner we have a healthy group and a disease group. We would like to merge the data from the 3 different scanners in order to classify the healthy subjects from the one who have the disease. The problem is that we can almost perfectly classify the subjects according to the scanner (e.g. the healthy subjects from scanner 1 and scanner 2). We are using a custom cross validation schema to account for the different scanners: when no hyper-parameter (SVM) optimization is performed, everything is straightforward. Problems arise when we would like to perform hyperparameter optimization: in this case we need to balance for the different scanner in the optimization phase as well. We also found a custom cv schema for this, but we are not able to pass it to GridSearchCV object. We would like to get something like the following: pipeline = Pipeline([('scl', StandardScaler()), ('sel', RFE(estimator,step=0.2)), ('clf', SVC(probability=True, random_state=42))]) param_grid = [{'sel__n_features_to_select':[22,15,10,2], 'clf__C': np.logspace(-3, 5, 100), 'clf__kernel':['linear']}] clf = GridSearchCV(pipeline, param_grid=param_grid, verbose=1, scoring='roc_auc', n_jobs= -1) # cv_final is the custom cv for the outer loop (9 folds) ii = 0 while ii < len(cv_final): # fit and predict clf.fit(data[?]], y[[?]]) predictions.append(clf.predict(data[cv_final[ii][1]])) # outer test data ii = ii + 1 We tried almost everything. When we define clf in the loop, we pass the -ith cv_nested as cv argument, and we fit it on the training data of the -ith custom_cv fold, we get an "Too many values to unpack" error. On the other end, when we try to pass the nested -ith cv fold as cv argument for clf, and we call fit on the same cv_nested fold, we get an "Index out of bound" error. Two questions: 1) Is there any workaround to avoid the split when clf is called without a cv argument? 2) We suppose that for hyperparameter optimization the test data is removed from the dataset and a new dataset is created. Is this true? In this case we only have to adjust the indices accordingly Thank your for your time and sorry for the long text Ludovico -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20170227/e80777cb…> ------------------------------ Message: 3 Date: Mon, 27 Feb 2017 11:27:24 -0500 From: Sebastian Raschka <se.raschka(a)gmail.com> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] Control over the inner loop in GridSearchCV Message-ID: <FC403FD1-9A00-424A-8453-9D60FE176C92(a)gmail.com> Content-Type: text/plain; charset=utf-8 Hi, Ludovico, what format (shape) is data in? Are these the arrays from a Kfold iterator? In this case, the ?question marks? in your code snippet should simply be the train and validation subset indices generated by the KFold generator. E.g., skfold = StratifiedKFold(y=y_train, n_folds=5, shuffle=True, random_state=1) for outer_train_idx, outer_valid_idx in skfold: ? gridsearch_object.fit(X_train[outer_train_idx], y_train[outer_train_idx]) > > On the other end, when we try to pass the nested -ith cv fold as cv argument for clf, and we call fit on the same cv_nested fold, we get an "Index out of bound" error. > Two questions: Are you using an version older than scikit-learn 0.18? Techically, the GridSearchCV, RandomizedSearchCV, cross_val_score ? should all support iterables that of train_ and test_indices e.g.: outer_cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=1) for name, gs_est in sorted(gridcvs.items()): nested_score = cross_val_score(gs_est, X=X_train, y=y_train, cv=outer_cv, n_jobs=1) Best, Sebastian > On Feb 27, 2017, at 9:27 AM, Ludovico Coletta <ludo25_90(a)hotmail.com> wrote: > > Dear Scikit experts, > > we am stucked with GridSearchCV. Nobody else was able/wanted to help us, we hope you will. > > We are analysing neuroimaging data coming from 3 different MRI scanners, where for each scanner we have a healthy group and a disease group. We would like to merge the data from the 3 different scanners in order to classify the healthy subjects from the one who have the disease. > > The problem is that we can almost perfectly classify the subjects according to the scanner (e.g. the healthy subjects from scanner 1 and scanner 2). We are using a custom cross validation schema to account for the different scanners: when no hyper-parameter (SVM) optimization is performed, everything is straightforward. Problems arise when we would like to perform hyperparameter optimization: in this case we need to balance for the different scanner in the optimization phase as well. We also found a custom cv schema for this, but we are not able to pass it to GridSearchCV object. We would like to get something like the following: > > pipeline = Pipeline([('scl', StandardScaler()), > ('sel', RFE(estimator,step=0.2)), > ('clf', SVC(probability=True, random_state=42))]) > > > param_grid = [{'sel__n_features_to_select':[22,15,10,2], > 'clf__C': np.logspace(-3, 5, 100), > 'clf__kernel':['linear']}] > > clf = GridSearchCV(pipeline, > param_grid=param_grid, > verbose=1, > scoring='roc_auc', > n_jobs= -1) > > # cv_final is the custom cv for the outer loop (9 folds) > > ii = 0 > > while ii < len(cv_final): > # fit and predict > > clf.fit(data[?]], y[[?]]) > predictions.append(clf.predict(data[cv_final[ii][1]])) # outer test data > ii = ii + 1 > > We tried almost everything. When we define clf in the loop, we pass the -ith cv_nested as cv argument, and we fit it on the training data of the -ith custom_cv fold, we get an "Too many values to unpack" error. On the other end, when we try to pass the nested -ith cv fold as cv argument for clf, and we call fit on the same cv_nested fold, we get an "Index out of bound" error. > Two questions: > 1) Is there any workaround to avoid the split when clf is called without a cv argument? > 2) We suppose that for hyperparameter optimization the test data is removed from the dataset and a new dataset is created. Is this true? In this case we only have to adjust the indices accordingly > > Thank your for your time and sorry for the long text > Ludovico > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... ------------------------------ Subject: Digest Footer _______________________________________________ scikit-learn mailing list scikit-learn(a)python.org https://mail.python.org/mailman/listinfo/scikit-learn scikit-learn Info Page - Python<https://mail.python.org/mailman/listinfo/scikit-learn> mail.python.org To see the collection of prior postings to the list, visit the scikit-learn Archives. Using scikit-learn: To post a message to all the list members ... ------------------------------ End of scikit-learn Digest, Vol 11, Issue 29 ********************************************

3 2

GSoC 2017
by Gael Varoquaux Feb. 27, 2017

Feb. 27, 2017

Hi, Students have been inquiring about the GSoC (Google Summer of Code) with scikit-learn, and the core team has been quite silent about team. I am happy to announce that we will be taking part in the scikit-learn again. The reason that we decided to do this is to give a chance to the young, talented, and motivated students. Importantly, our most limiting resource is the time of our experienced developers. This is clearly visible from the number of pending pull requests. Hence, we need students to be very able and independent. This of course means that they will be getting supervision from mentors. Such supervision is crucial for moving forward with a good project, that delivers mergeable code. However, we will need the students to be very good at interacting efficiently with the mentors. Also, I should stress that we will be able to take only a very few numbers of students. With that said, let me introduce the 2017 GSoC for scikit-learn. We have set up a wiki page which summarizes the experiences from last year and the ideas for this year: https://github.com/scikit-learn/scikit-learn/wiki/Google-summer-of-code-(GS… Interested students should declare their interest on the mailing list, and discuss with possible mentors here. Factors of success will be * careful work on a good proposal, that takes on of the ideas on the wiki but breaks it down in a realistic plan with multiple steps and shows a good understanding of the problem. * demonstration of the required skillset via successful pull requests in scikit-learn. Cheers, Gaël -- Gael Varoquaux Researcher, INRIA Parietal NeuroSpin/CEA Saclay , Bat 145, 91191 Gif-sur-Yvette France Phone: ++ 33-1-69-08-79-68 http://gael-varoquaux.info http://twitter.com/GaelVaroquaux

5 6