Mailman 3 June 2017 - scikit-learn

Agglomerative Clustering without knowing number of clusters
by Ariani A July 13, 2017

July 13, 2017

I want to perform agglomerative clustering, but I have no idea of number of clusters before hand. But I want that every cluster has at least 40 data points in it. How can I apply this to sklearn.agglomerative clustering? Should I use dendrogram and cut it somehow? I have no idea how to relate dendrogram to this and cutting it out. Any help will be appreciated!

2 6

Construct the microclusters using a CF-Tree
by Sema Atasever July 6, 2017

July 6, 2017

Hi all, I want to ask you about clustering usign Birch clustering algorithm. I have a *distance matrix* n*n M where M_ij is the distance between object_i and object_j.(You can see file format in the attachment). I want to cluster them using Birch clustering algorithm. Does this method have 'precomputed' option. I needed train an SVM on the centroids of the microclusters so *How can i get the centroids of the microclusters?* Any help would be highly appreciated. *Birch code:* from sklearn.cluster import Birch from io import StringIO import numpy as np X=np.loadtxt(open("C:\dm.txt", "rb"), delimiter="\t") brc = Birch(branching_factor=50, n_clusters=3, threshold=0.5,compute_labels=True,copy=True) brc.fit(X) brc.predict(X) print(brc.predict(X)) Any help would be highly appreciated.

2 5

Machine learning for PU data
by Ruchika Nayyar July 5, 2017

July 5, 2017

Hi All, I am a scikit-learn user and have a question for the community, if anyone has applied any available machine learning algorithms in the scikit-learn package for data with positive and unlabeled class only? If so would you share some insight with me. I understand this could be a broader topic but I am new to analyzing PU data and hence can use some help. Thanks, Ruchika

3 2

Scikit-learn workshop and sprint at EuroScipy 2017 in Erlangen
by Olivier Grisel July 3, 2017

July 3, 2017

Hi all, FYI I have just submitted a 90 min tutorial on scikit-learn to the EuroScipy CFP. If anybody is interested in co-teaching / TA-ing this workshop please let me know. I also plan to stay for the one-day sprint to help people make their first contribution to the project. Last year we had great fun and the sprint was very productive. Registration is now open: https://www.euroscipy.org/2017/ 10th European Conference on Python in Science Location: Erlangen August 28-29 (Mon, Tue) Tutorials / Workshops August 30 - 31 (Wed, Thu) Main conference and posters September 1 (Fri) Sprints See you in Erlangen! -- Olivier http://twitter.com/ogrisel - http://github.com/ogrisel

2 3

Scikit-learn at Data Intelligence this past weekend
by Francois Dion June 30, 2017

June 30, 2017

This past weekend was the Numfocus sponsored Data Intelligence conference at Capital One, in Mclean, Virginia (close to Washington DC for those not familiar with the US geography). A few presentations mentioned/used scikit-learn, including Ben Bengfort's Visual Pipelines ( http://data-intelligence.ai/presentations/13 ), Zachary Beaver's Airflow + Scikit-Learn ( http://data-intelligence.ai/presentations/19 ) and Pramit Choudary's Learning to Learn Model Behavior ( http://data-intelligence.ai/presentations/22 ), to name a few. I presented "Seeking Exotics" on Sunday ( http://data-intelligence.ai/presentations/21), on anomalous and erroneous data, and how statistics, visualizations and scikit-learn can help (covered PCA, truncatedSVD, t-sne, ellipticenvelope, one class classifiers and scikit-learn related imbalanced-learn and sk-sos). One of the slide I had up resonated quite a bit with the audience, both in person and on social media: https://twitter.com/tnfilipiak/status/878999245076008960 The notebooks are on github: https://github.com/fdion/seeking_exotics Francois -- @f_dion - https://about.me/francois.dion - https://www.linkedin.com/in/francois-dion-8b639b79/

4 3

Fwd: [SciPy-User] EuroSciPy 2017 call for contributions - extension of deadline
by Nelle Varoquaux June 30, 2017

June 30, 2017

Hi everyone, I thought some of you might be interested in this dead line extension. Cheers, N ---------- Forwarded message ---------- From: Pierre de Buyl <pierre.debuyl(a)kuleuven.be> Date: 26 June 2017 at 03:49 Subject: [SciPy-User] EuroSciPy 2017 call for contributions - extension of deadline To: scipy-user(a)python.org, numpy-discussion(a)python.org (Apologies if you receive multiple copies of this message) 10th European Conference on Python in Science August 28 - September 1, 2017 in Erlangen, Germany The Call for Papers is extended to July 02, 2017 23:00 CEST Description: The EuroSciPy meeting is a cross-disciplinary gathering focused on the use and development of the Python language in scientific research. This event strives to bring together both users and developers of scientific tools, as well as academic research and state of the art industry. Erlangen is one of Germany's major science hubs and located north of Munich (90 minutes by train). The Call for Papers is extended to July 02, 2017 23:00 CEST Regards, The EuroSciPy team https://www.euroscipy.org/2017/ _______________________________________________ SciPy-User mailing list SciPy-User(a)python.org https://mail.python.org/mailman/listinfo/scipy-user

3 2

Agglomerative clustering
by Ariani A June 30, 2017

June 30, 2017

I have some data and also the pairwise distance matrix of these data points. I want to cluster them using Agglomerative clustering. I readthat in sklearn, we can have 'precomputed' as affinity and I expect it is the distance matrix. But I could not find any example which uses precomputed affinity and a custom distance matrix. Any help will be highly appreciated. Best, -Noushin

2 1

Fwd: sample_weight parameter is not split when used in GridSearchCV
by Manuel Castejón Limas June 27, 2017

June 27, 2017

Dear all, I posted the full question on StackOverflow and as it contains some figures I refer you to that post. https://stackoverflow.com/questions/44661926/sample- weight-parameter-shape-error-in-scikit-learn-gridsearchcv/44662285#44662285 I currently believe that this issue is a bug and I opened an issue on GitHub. To sum up, the issue is that GridSearchCV does not handle the splitting of the sample_weight vector during cross validation. Nota bene: cross_val_score seems to handle the splitting OK, this issue seems to occurr only in GridSearchCV. Any comments enlightening me and showing me how wrong I am are most welcome.

5 10

[Feature] drop_one in one hot encoder
by Parminder Singh June 25, 2017

June 25, 2017

Hy Sci-kittens! :-) I was doing machine learning a-z course on Udemy, there they told that every time one-hot encoding is done, one of the columns should be dropped as it is like doubling same category twice and redundant to model. I thought if instead of having user find the index and drop it after preprocessing, OneHotEncoder had a drop_one variable, and it automatically removed the last column. What are your thoughts about this? I am new to this community, would like to contribute this myself if it is possible addition. Thanks, Trion129

3 2

Query about use of standard deviation on tree feature_importances_ in demo plot_forest_importances.html
by Ian Ozsvald June 24, 2017

June 24, 2017

Hi all. I'm looking at the code behind one of the tree ensemble demos: http://scikit-learn.org/stable/auto_examples/ensemble/plot_forest_importanc… and I'm unsure about the error bars. They are calculated using the standard deviation of the feature_importances_ attribute across trees. Can we depend on this being a Normal distribution? I'm wondering if the plot tells enough of the story to be genuinely useful? I don't have a strong belief in the likely distribution of feature_importances_, I haven't dug into how the feature importances are calculated (frankly I'm a bit lost here). I know that on a RF Regression case I'm working on I can see unimodal and bimodal feature importance distributions - this came up on a discussion on the yellowbrick sklearn visualisation package: https://github.com/DistrictDataLabs/yellowbrick/pull/195 I don't know what is "normal" for feature importances and if they look different between classification tasks (as in the plot_forest_importances demo) and regression tasks. Maybe I've got an outlier in my task? If I use the provided demo code then my error bars can go negative, so that feels unhelpful. Does anyone have an opinion? Perhaps more importantly - is a visual indication of the spread of feature importances in an ensemble actually a useful thing to plot? Does it serve a diagnostic value? I saw Sebastian Raschka's reference to Gilles Louppe et al.'s NIPS paper (in here, 2016-05-17) on variable importances, I'll dig into that if nobody has a strong opinion. BTW Sebastian - thanks for writing your book. Cheers, Ian. -- Ian Ozsvald (Data Scientist, PyDataLondon co-chair) ian(a)IanOzsvald.com http://IanOzsvald.com http://ModelInsight.io http://twitter.com/IanOzsvald

2 2