Mailman 3 March 2020 - scikit-learn

[GridSearchCV] Reduction of elapsed time at the second interation
by Pedro Cardoso May 30, 2020

May 30, 2020

Hello fellows, i am knew at slkearn and I have a question about GridSearchCV: I am running the following code at a jupyter notebook : ----------------------*code*------------------------------- opt_models = dict() for feature in [features1, features2, features3, features4]: cmb = CMB(x_train, y_train, x_test, y_test, feature) cmb.fit() cmb.predict() opt_models[str(feature)]=cmb.get_best_model() ------------------------------------------------------- The CMB class is just a class that contains different classification models (SVC, decision tree, etc...). When cmb.fit() is running, a gridSearchCV is performed at the SVC model (which is within the cmb instance) in order to tune the hyperparameters C, gamma, and kernel. The SCV model is implemented using the sklearn.svm.SVC class. Here is the output of the first and second iteration of the for loop: ---------------------*output*------------------------------------- -> 1st iteration Fitting 5 folds for each of 12 candidates, totalling 60 fits [Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers. [Parallel(n_jobs=-1)]: Done 1 tasks | elapsed: 6.1s [Parallel(n_jobs=-1)]: Done 2 tasks | elapsed: 6.1s [Parallel(n_jobs=-1)]: Done 3 tasks | elapsed: 6.1s [Parallel(n_jobs=-1)]: Done 4 tasks | elapsed: 6.2s [Parallel(n_jobs=-1)]: Done 5 tasks | elapsed: 6.2s [Parallel(n_jobs=-1)]: Done 6 tasks | elapsed: 6.2s [Parallel(n_jobs=-1)]: Done 7 tasks | elapsed: 6.2s [Parallel(n_jobs=-1)]: Done 8 tasks | elapsed: 6.2s [Parallel(n_jobs=-1)]: Done 9 tasks | elapsed: 6.2s [Parallel(n_jobs=-1)]: Done 10 tasks | elapsed: 6.2s [Parallel(n_jobs=-1)]: Done 11 tasks | elapsed: 6.2s [Parallel(n_jobs=-1)]: Done 12 tasks | elapsed: 6.3s [Parallel(n_jobs=-1)]: Done 13 tasks | elapsed: 6.3s [Parallel(n_jobs=-1)]: Done 14 tasks | elapsed: 6.3s [Parallel(n_jobs=-1)]: Done 15 tasks | elapsed: 6.4s [Parallel(n_jobs=-1)]: Done 16 tasks | elapsed: 6.4s [Parallel(n_jobs=-1)]: Done 17 tasks | elapsed: 6.4s [Parallel(n_jobs=-1)]: Done 18 tasks | elapsed: 6.4s [Parallel(n_jobs=-1)]: Done 19 tasks | elapsed: 6.5s [Parallel(n_jobs=-1)]: Done 20 tasks | elapsed: 6.5s [Parallel(n_jobs=-1)]: Done 21 tasks | elapsed: 6.5s [Parallel(n_jobs=-1)]: Done 22 tasks | elapsed: 6.6s [Parallel(n_jobs=-1)]: Done 23 tasks | elapsed: 6.7s [Parallel(n_jobs=-1)]: Done 24 tasks | elapsed: 6.7s [Parallel(n_jobs=-1)]: Done 25 tasks | elapsed: 6.7s [Parallel(n_jobs=-1)]: Done 26 tasks | elapsed: 6.8s [Parallel(n_jobs=-1)]: Done 27 tasks | elapsed: 6.8s [Parallel(n_jobs=-1)]: Done 28 tasks | elapsed: 6.9s [Parallel(n_jobs=-1)]: Done 29 tasks | elapsed: 6.9s [Parallel(n_jobs=-1)]: Done 30 tasks | elapsed: 6.9s [Parallel(n_jobs=-1)]: Done 31 tasks | elapsed: 7.0s [Parallel(n_jobs=-1)]: Done 32 tasks | elapsed: 7.0s [Parallel(n_jobs=-1)]: Done 33 tasks | elapsed: 7.0s [Parallel(n_jobs=-1)]: Done 34 tasks | elapsed: 7.0s [Parallel(n_jobs=-1)]: Done 35 tasks | elapsed: 7.1s [Parallel(n_jobs=-1)]: Done 36 tasks | elapsed: 7.1s [Parallel(n_jobs=-1)]: Done 37 tasks | elapsed: 7.2s [Parallel(n_jobs=-1)]: Done 38 tasks | elapsed: 7.2s [Parallel(n_jobs=-1)]: Done 39 tasks | elapsed: 7.2s [Parallel(n_jobs=-1)]: Done 40 tasks | elapsed: 7.2s [Parallel(n_jobs=-1)]: Done 41 tasks | elapsed: 7.3s [Parallel(n_jobs=-1)]: Done 42 tasks | elapsed: 7.3s [Parallel(n_jobs=-1)]: Done 43 tasks | elapsed: 7.3s [Parallel(n_jobs=-1)]: Done 44 tasks | elapsed: 7.4s [Parallel(n_jobs=-1)]: Done 45 tasks | elapsed: 7.4s [Parallel(n_jobs=-1)]: Done 46 tasks | elapsed: 7.5s -> 2nd iteration Fitting 5 folds for each of 12 candidates, totalling 60 fits [Parallel(n_jobs=-1)]: Using backend LokyBackend with 4 concurrent workers. [Parallel(n_jobs=-1)]: Done 1 tasks | elapsed: 0.0s [Parallel(n_jobs=-1)]: Batch computation too fast (0.0260s.) Setting batch_size=14. [Parallel(n_jobs=-1)]: Done 2 tasks | elapsed: 0.0s [Parallel(n_jobs=-1)]: Done 3 tasks | elapsed: 0.0s [Parallel(n_jobs=-1)]: Done 4 tasks | elapsed: 0.0s [Parallel(n_jobs=-1)]: Done 5 tasks | elapsed: 0.0s [Parallel(n_jobs=-1)]: Done 60 out of 60 | elapsed: 0.7s finished --------------------------------------------------------------------------------------------------------------------- As you can see, the first iteration gets a elapsed time much larger than the 2nd iteration. Does it make sense? I am afraid that the model is doing some kind of cache or shortcut from the 1st iteration, and consequently could decrease the model training/performance? I already read the sklearn documentation and I didn't saw any warning/note about this kind of behaviour. Thank you very much for your time :)

2 2

Number of informative features vs total number of features
by Benoît Presles April 3, 2020

April 3, 2020

Dear sklearn users, I did some supervised classification simulations with the make_classification function from sklearn increasing the number of informative features from 1 out of 40 to 40 out of 40 (100%). I did not generate any repeated or redundant features. I fixed the number of classes to two and the number of clusters per class to one. I split the dataset 100 times using the StratifiedShuffleSplit function into two subsets: a training set and a test set (80% - 20%). I performed a logistic regression and calculated training and testing accuracies and averaged the results over the 100 splits leading to a mean training accuracy and a mean testing accuracy. I was expecting to get an increasing accuracy score as a function of informative features for both the training and the test sets. On the contrary, I have got the best training and test scores for one informative feature. Why do I get these results ? Thanks for your help, Best regards, Ben Below the simulation code I have written: import numpy as np from sklearn.datasets import make_classification from sklearn.model_selection import StratifiedShuffleSplit from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression from sklearn.metrics import accuracy_score import matplotlib.pyplot as plt RANDOM_SEED = 4 n_inf = np.array([1, 5, 10, 15, 20, 25, 30, 35, 40]) mean_training_score_array = np.array([]) mean_testing_score_array = np.array([]) for n_inf_value in n_inf: X, y = make_classification(n_samples=2500, n_features=40, n_informative=n_inf_value, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=1, random_state=RANDOM_SEED, shuffle=False) # print('Simulated data - number of informative features = ' + str(n_inf_value)) # sss = StratifiedShuffleSplit(n_splits=100, test_size=0.2, random_state=RANDOM_SEED) training_score_array = np.array([]) testing_score_array = np.array([]) for train_index_split, test_index_split in sss.split(X, y): X_split_train, X_split_test = X[train_index_split], X[test_index_split] y_split_train, y_split_test = y[train_index_split], y[test_index_split] scaler = StandardScaler() X_split_train = scaler.fit_transform(X_split_train) X_split_test = scaler.transform(X_split_test) lr = LogisticRegression(fit_intercept=True, max_iter=1e9, verbose=0, random_state=RANDOM_SEED, solver='lbfgs', tol=1e-6, C=10) lr.fit(X_split_train, y_split_train) y_pred_train = lr.predict(X_split_train) y_pred_test = lr.predict(X_split_test) accuracy_train_score = accuracy_score(y_split_train, y_pred_train) accuracy_test_score = accuracy_score(y_split_test, y_pred_test) training_score_array = np.append(training_score_array, accuracy_train_score) testing_score_array = np.append(testing_score_array, accuracy_test_score) mean_training_score_array = np.append(mean_training_score_array, np.average(training_score_array)) mean_testing_score_array = np.append(mean_testing_score_array, np.average(testing_score_array)) # print('mean_training_score_array=' + str(mean_training_score_array)) print('mean_testing_score_array=' + str(mean_testing_score_array)) # plt.plot(n_inf, mean_training_score_array, 'r', label='mean training score') plt.plot(n_inf, mean_testing_score_array, 'g', label='mean testing score') plt.xlabel('number of informative features out of 40') plt.ylabel('accuracy') plt.legend() plt.show()

2 2

A basic question about kmeans algorithms elkan and llyod
by MC_George123＠hotmail.com March 31, 2020

March 31, 2020

Hi admins, My team is working on optimization on scikit-learn staff now. When it comes to kmeans, I find there are two algorithms, one of which is lloyd and the other is elkan, which is the optimized one for lloyd using triangle inequality. In the older version of scikit-learn, elkan only supports dense dataset instead of sparse one. And in the latest version, elkan supports both type of datasets. So there is a question why both two algorithms are kept in kmeans since they do the almost same thing and elkan is a optimized one for lloyd. Are there any precision difference between two algorithms and how can I decide what algorithm to use? Best regards, George Fan

4 5

Analysis of sklearn and other python libraries on github by MS team
by Andreas Mueller March 30, 2020

March 30, 2020

Hey all. There's a pretty cool paper by a team at MS that analyses public github repos for their use of the sklearn and related libraries: https://arxiv.org/abs/1912.09536 Thought it might be of interest. Cheers, Andy

3 4

Monthly meetings
by Nicolas Hug March 30, 2020

March 30, 2020

Hi all, The next scikit-learn monthly meeting will take place on Monday (https://www.timeanddate.com/worldclock/meetingdetails.html?year=2020&month=… <https://www.google.com/url?q=https://www.timeanddate.com/worldclock/meeting…>) While these meetings are mainly for core-devs to discuss the current topics, we're also happy to welcome non-core devs and other projects maintainers! Feel free to join. *Location:* Join Zoom Meeting https://anaconda.zoom.us/j/947129165?pwd=dEFZNHM0ZFBiQWlDYlJlRW1EaHg2QT09 <https://www.google.com/url?q=https://anaconda.zoom.us/j/947129165?pwd%3DdEF…> Meeting ID: 947 129 165 Password: 586745 Thanks, Nicolas

3 2

Final CFP CodiEsp: Clinical Case Coding Task (eHealth CLEF 2020)
by Martin Krallinger March 19, 2020

March 19, 2020

*** Call for Participation CodiEsp: Clinical Case Coding Task (eHealth CLEF 2020) *** * *CodiEsp (eHealth CLEF– Multilingual Information Extraction) Shared Task on automatic assignment of ICD10 codes (procedures, diagnosis) track at CLEF 2020* http://temu.bsc.es/codiesp Plan TL Award for the CodiEsp Track The CodiEsp sub-tracks: *1.CodiEsp Diagnosis Coding *sub-task* (CodiEsp-D)*: will require automatic ICD10-CM [CIE10 Diagnóstico] code assignment to each clinical case document. *2.CodiEsp Procedure Coding *sub-task* (CodiEsp-P):* will require automatic ICD10-PCS [CIE10 Procedimiento] code assignment to each clinical case document. *3.CodiEsp Explainable AI *exploratory sub-task* (CodiEsp-X).* Systems are required to extract the evidence text supporting the predicted codes (both ICD10-CM and ICD10-PCS). *Task description* Clinical coding essentially requires the transformation (or classification) of medical texts into a structured or coded format using internationally recognized class codes. These codes describe a patient’s diagnosis or treatment. Clinical coding is critical for standardizing electronic clinical records; enable aetiology studies, monitor health trends, carry out epidemiology studies, clinical and biomedical research, assist clinical decision-making or even reimbursement. As part of the eHealth CLEF (http://clef-ehealth.org) Multilingual Information Extraction Shared Task we organize* CodiEsp: Clinical Case Coding Task (http://temu.bsc.es/codiesp <http://temu.bsc.es/codiesp>). *The CodiEsp task will address the automatic extraction and assignment of clinical coding (diagnosis and procedures) to clinical case documents in Spanish. To enable participation of researches around the world, in addition to the basic data in Spanish, we will also publish versions of the training, development, and test set *automatically translated into English*. Participating systems will be asked to automatically assign ICD10 codes (or CIE-10, in Spanish) to clinical case documents. Evaluation is done through comparison to manually assigned ICD10 codes. *Publications and workshop* As in previous eHealth CLEF efforts, there will be an *evaluation workshop allocated at CLEF 2020* where participating teams can present their systems and results. Moreover, participating teams will be invited to submit their system description papers for publication at the *CLEF 2020 Working Notes proceedings*. For previous working notes see: http://ceur-ws.org/Vol-2125/ *CodiEsp awards* There will we three awards for the top-scoring teams promoted by the Spanish Plan for the Advancement of Language Technology (Plan TL) and the Barcelona Supercomputing Center (BSC). -------------------------------------- *Participation and useful info* -------------------------------------- 1. CodiEsp web, info & detailed description:http://temu.bsc.es/codiesp/ 2. Registration for CodiEsp (Multilingual Information Extraction eHealth track):http://temu.bsc.es/codiesp/index.php/registration/ 3. Datasets:https://zenodo.org/record/3693570 <https://zenodo.org/record/3693570#.Xl5bK9-YU5k> 4. Additional training resources:https://doi.org/10.5281/zenodo.3606662 ------------------------ *Main CodiEsp Track organizers* ------------------------ - *Martin Krallinger*, Barcelona Supercomputing Center. - *Antonio Miranda*, Barcelona Supercomputing Center. - *Aitor Gonzalez-Agirre*, Barcelona Supercomputing Center. - *Marta Villegas*, Barcelona Supercomputing Center. - *Jordi Armengol*, Barcelona Supercomputing Center. ------------------------ *Important Dates* ------------------------ Jan 13: Training and development set release March 2: Test and background set release May 3: End of evaluation May 5: Results notified May 24: Paper submission Jun 28: Camera-ready paper submission Sep 22-25: CLEF 2020 Conference (Thessaloniki, Greece)

1 0

transfer learning doubt
by Praneet Singh March 19, 2020

March 19, 2020

I am training a SGD Classifier with some training dataset which is temporary and will be lost after sometime. So I am planning to save the model in pickle file and reuse it and train again with some another dataset that arrives. But It forgets the previously learned data. As far as I researched in google, tensorflow model allows transfer learning and not forgetting the previous learning but is there any other way with sklearn model to achieve this?? any help would be appreciated

3 2

The Coronavirus Tech Handbook
by Juan BC March 18, 2020

March 18, 2020

Sorry for the offtopic https://coronavirustechhandbook.com/ <<<< The Coronavirus Tech Handbook provides a space for technologists, specialists, civic organisations and public & private institutions to collaborate on a rapid and sophisticated response to the coronavirus outbreak. It is a dynamic resource with many hundreds of contributors that is evolving very quickly. -- Juan B Cabral

1 0

Announcing the 2020 John Hunter Excellence in Plotting Contest
by Nelle Varoquaux March 17, 2020

March 17, 2020

Dear all, I apologize for the cross-posting. In memory of John Hunter, we are pleased to announce the John Hunter Excellence in Plotting Contest for 2020. This open competition aims to highlight the importance of data visualization to scientific progress and showcase the capabilities of open source software. Participants are invited to submit scientific plots to be judged by a panel. The winning entries will be announced and displayed at SciPy 2020 or announced in the John Hunter Excellence in Plotting Contest website and youtube channel. John Hunter’s family are graciously sponsoring cash prizes for the winners in the following amounts: - 1st prize: $1000 - 2nd prize: $750 - 3rd prize: $500 - Entries must be submitted by June 1st to the form at https://forms.gle/SrexmkDwiAmDc7ej7 - Winners will be announced at Scipy 2020 in Austin, TX or publicly on the John Hunter Excellence in Plotting Contest website and youtube channel - Participants do not need to attend the Scipy conference. - Entries may take the definition of “visualization” rather broadly. Entries may be, for example, a traditional printed plot, an interactive visualization for the web, a dashboard, or an animation. - Source code for the plot must be provided, in the form of Python code and/or a Jupyter notebook, along with a rendering of the plot in a widely used format. The rendering may be, for example, PDF for print, standalone HTML and Javascript for an interactive plot, or MPEG-4 for a video. If the original data can not be shared for reasons of size or licensing, "fake" data may be substituted, along with an image of the plot using real data. - Each entry must include a 300-500 word abstract describing the plot and its importance for a general scientific audience. - Entries will be judged on their clarity, innovation and aesthetics, but most importantly for their effectiveness in communicating a real-world problem. Entrants are encouraged to submit plots that were used during the course of research or work, rather than merely being hypothetical. - SciPy and the John Hunter Excellence in Plotting Contest organizers reserves the right to display any and all entries, whether prize-winning or not, at the conference, use in any materials or on its website, with attribution to the original author(s). - Past entries can be found at https://jhepc.github.io/ - Questions regarding the contest can be sent to jhepc.organizers(a)gmail.com John Hunter Excellence in Plotting Contest Co-Chairs Madicken Munk Nelle Varoquaux

1 0

Study on annotation of design and implementation choices, and of technical debt
by Gianmarco Fucci March 16, 2020

March 16, 2020

Dear all, As software engineering research teams at the University of Sannio (Italy) and Eindhoven University of Technology (The Netherlands) we are interested in investigating the protocol used by developers while they have to annotate implementation and design choices during their normal development activities. More specifically, we are looking at whether, where and what kind of annotations developers usually use trying to be focused more on those annotations mainly aimed at highlighting that the code is not in the right shape (e.g., comments for annotating delayed or intended work activities such as TODO, FIXME, hack, workaround, etc). In the latter case, we are looking at what is the content of the above annotations, as well as how they usually behave while evolving the code that has been previously annotated. When answering the survey, in case your annotation practices are different in different open source projects you may contribute, please refer to how you behave for the projects where you have been contacted. Filling out the survey will take about 5 minutes. Please note that your identity and personal data will not be disclosed, while we plan to use the aggregated results and anonymized responses as part of a scientific publication. If you have any questions about the questionnaire or our research, please do not hesitate to contact us. You can find the survey link here: https://forms.gle/NxdVXiZQSmQ15U4T8 Thanks and regards, Gianmarco Fucci (gianmarcofucci94(a)gmail.com) Fiorella Zampetti (fzampetti(a)unisannio.it) Alexander Serebrenik (a.serebrenik(a)tue.nl) Massimiliano Di Penta (dipenta(a)unisannio.it)

1 0