Mailman 3 October 2017 - scikit-learn

Topic for thesis work on scikit learn
by Gaurav Dhingra Dec. 31, 2017

Dec. 31, 2017

Hi everyone, I am a final year (5th year) undergraduate Applied Mathematics student in India. I am thinking of doing my final year thesis by doing some work (coding part) on scikit learn, so I was thinking if anyone could tell me if there are available topics (not necessarily names of those topics) that I could work on being an undergraduate student? I would want to expand upon this in December when my exams will be over. But in the mean time would want to take a step in that direction by just knowing if there will be available topics that I could work on. It could be the case that available topics are not so easy for an undergraduate, still in that case I would like to do some research on the topics first. -- Best, Gaurav Dhingra (sent from Thunderbird email client)

2 4

laoding multi-targets
by Luigi Lomasto Oct. 26, 2017

Oct. 26, 2017

Hi, I'm writing to ask a question. Can I load a dataset multilabels with load_files function? For example, I have a dataset like this: folder folder1 txt1 txt2 folder2 txt2 txt3 So, item txt2 have two labels, folder1 and foder2. How I can load mutli-targets? Thanks for answer. Best regards, Luigi

2 1

Interested to Contribute to Scikit Learn
by Nikhil Rayaprolu Oct. 24, 2017

Oct. 24, 2017

Hello EveryOne, I am Nikhil Rayaprolu, an undergraduate student from India, I would like to contribute to scikit-learn. Can anyone help me by giving me some critical issue, that could be a good contribution to scikit learn. I have come across Parallel decision tree building in proposed projects, so I would like to know the status of it and if that could be contributed I am happy to contribute to the issue. Thank You Nikhil Rayaprolu

2 1

[ANN] scikit-learn 0.19.1 is out!
by Andreas Mueller Oct. 23, 2017

Oct. 23, 2017

Hey everybody. We just released 0.19.1, fixing some issues and bugs in the last release. It's highly recommended you upgrade your installation. The new release is available via pip, conda (main) and conda-forge. A big thank you to everybody who contributed, in particular Joel (@jnothman)! The release includes several improvements and fixes to the model_selection and pipeline modules, and t-SNE. You can find the full changelog here: http://scikit-learn.org/stable/whats_new.html#version-0-19-1 Happy learning! Andy

3 2

How to get centroids from SciPy's hierarchical agglomerative clustering?
by Sema Atasever Oct. 23, 2017

Oct. 23, 2017

Dear scikit-learn members, I am using SciPy's hierarchical agglomerative clustering methods to cluster a 1000 x 22 matrix of features, after clustering my data set with scipy.cluster.hierarchy.linkage and and assigning each sample to a cluster, I can't seem to figure out how to get the centroid from the resulting clusters. I would like to extract one element or a few out of each cluster, which is the closest to that cluster's centroid. Below follows my code: *D=np.loadtxt(open("C:\dataset.txt", "rb"), delimiter=";")* *Y = hierarchy.linkage(D, 'ward')* *assignments = hierarchy.fcluster(Y, 5, criterion="maxclust")* I am taking my matrix of features, computing the euclidean distance between them, and then passing them onto the hierarchical clustering method. From there, I am creating flat clusters, with a maximum of 5 clusters Now, based on the flat clusters *assignments*, how do I get the 1 x 22 centroid that represents each flat cluster? Best.

3 3

question for using GridSearchCV on LocalOutlierFactor
by Lifan Xu Oct. 22, 2017

Oct. 22, 2017

Hi, I was trying to train a model for anomaly detection. I only have the normal data which are all labeled as 1. Here is my code: clf = sklearn.model_selection.GridSearchCV(sklearn.neighbors.LocalOutlierFactor(), parameters, scoring="accuracy", cv=kfold, n_jobs=10) clf.fit(vectors, labels) But it complains "AttributeError: 'LocalOutlierFactor' object has no attribute 'predict'". It looks like LocalOutlierFactor only has fit_predict(), but no predict(). My question is will predict() be implemented? Thanks!

5 5

How compute weighted accuracy for multi-class classification?
by Yaser Norouzzadeh Oct. 19, 2017

Oct. 19, 2017

I do multi-class classification on unbalanced classes. I'm using SGDClassifier(), GradientBoostingClassifier(), RandomForestClassifier(), and LogisticRegression()with class_weight='balanced'. To compare the results. it is required to compute the accuracy. I tried the following way to compute weighted accuracy: n_samples = len(y_train) weights_cof = float(n_samples)/(n_classes*np.bincount(data[target_label].as_matrix().astype(int))[1:]) sample_weights = np.ones((n_samples,n_classes)) * weights_cof print accuracy_score(y_test, y_pred, sample_weight=sample_weights) y_train is a binary array. So sample_weights has the same shape as y_train (n_samples, n_classes). When I run the script, I received the following error: Traceback (most recent call last): File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.3.2\helpers\pydev\pydevd.py", line 1596, in <module> globals = debugger.run(setup['file'], None, None, is_module) File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.3.2\helpers\pydev\pydevd.py", line 974, in run pydev_imports.execfile(file, globals, locals) # execute the script File "D:/Destiny/DestinyScripts/MainLocationAware.py", line 424, in <module> predict_country(featuresDF, score, featuresLabel, country_sample_size, 'gbc') File "D:/Destiny/DestinyScripts/MainLocationAware.py", line 313, in predict_country print accuracy_score(y_test, y_pred, sample_weight=sample_weights) File "C:\ProgramData\Anaconda2\lib\site-packages\sklearn\metrics\classification.py", line 183, in accuracy_score return _weighted_sum(score, sample_weight, normalize) File "C:\ProgramData\Anaconda2\lib\site-packages\sklearn\metrics\classification.py", line 108, in _weighted_sum return np.average(sample_score, weights=sample_weight) File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\lib\function_base.py", line 1124, in average "Axis must be specified when shapes of a and weights " TypeError: Axis must be specified when shapes of a and weights differ.

1 0

Re: [scikit-learn] scikit-learn Digest, Vol 19, Issue 37
by Ismael Lemhadri Oct. 18, 2017

Oct. 18, 2017

How about editing the various chunks of code concerned to add the option to scale the parameters, and set it by default to NOT scale? This would make what happens clear without the redundancy Andreas mentioned, and would add more convenience to the user shall they want to scale their data. On Tue, Oct 17, 2017 at 8:45 AM, <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: Unclear help file about sklearn.decomposition.pca (Raphael C) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 17 Oct 2017 16:44:55 +0100 > From: Raphael C <drraph(a)gmail.com> > To: Scikit-learn mailing list <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] Unclear help file about > sklearn.decomposition.pca > Message-ID: > <CAFHc1QZigBoA0erY2hwJht2kiAenp=QTQDSb3uc0Uzs5SoEC7Q@mail. > gmail.com> > Content-Type: text/plain; charset="UTF-8" > > How about including the scaling that people might want to use in the > User Guide examples? > > Raphael > > On 17 October 2017 at 16:40, Andreas Mueller <t3kcit(a)gmail.com> wrote: > > In general scikit-learn avoids automatic preprocessing. > > That's a convention to give the user more control and decrease surprising > > behavior (ostensibly). > > So scikit-learn will usually do what the algorithm is supposed to do, and > > nothing more. > > > > I'm not sure what the best way do document this is, as this has come up > with > > different models. > > For example the R wrapper of libsvm does automatic scaling, while we > apply > > the SVM. > > > > We could add "this model does not do any automatic preprocessing" to all > > docstrings, but that seems > > a bit redundant. We could add it to > > https://github.com/scikit-learn/scikit-learn/pull/9517, but > > that is probably not where you would have looked. > > > > Other suggestions welcome. > > > > > > On 10/16/2017 03:29 PM, Ismael Lemhadri wrote: > > > > Thank you all for your feedback. > > The initial problem I came with wasnt the definition of PCA but what the > > sklearn method does. In practice I would always make sure the data is > both > > centered and scaled before performing PCA. This is the recommended method > > because without scaling, the biggest direction could wrongly seem to > explain > > a huge fraction of the variance. > > So my point was simply to clarify in the help file and the user guide > what > > the PCA class does precisely to leave no unclarity to the reader. Moving > > forward I have now submitted a pull request on github as initially > suggested > > by Roman on this thread. > > Best, > > Ismael > > > > On Mon, 16 Oct 2017 at 11:49 AM, <scikit-learn-request(a)python.org> > wrote: > >> > >> Send scikit-learn mailing list submissions to > >> scikit-learn(a)python.org > >> > >> To subscribe or unsubscribe via the World Wide Web, visit > >> https://mail.python.org/mailman/listinfo/scikit-learn > >> or, via email, send a message with subject or body 'help' to > >> scikit-learn-request(a)python.org > >> > >> You can reach the person managing the list at > >> scikit-learn-owner(a)python.org > >> > >> When replying, please edit your Subject line so it is more specific > >> than "Re: Contents of scikit-learn digest..." > >> > >> > >> Today's Topics: > >> > >> 1. Re: 1. Re: unclear help file for sklearn.decomposition.pca > >> (Andreas Mueller) > >> 2. Re: 1. Re: unclear help file for sklearn.decomposition.pca > >> (Oliver Tomic) > >> > >> > >> ---------------------------------------------------------------------- > >> > >> Message: 1 > >> Date: Mon, 16 Oct 2017 14:44:51 -0400 > >> From: Andreas Mueller <t3kcit(a)gmail.com> > >> To: scikit-learn(a)python.org > >> Subject: Re: [scikit-learn] 1. Re: unclear help file for > >> sklearn.decomposition.pca > >> Message-ID: <35142868-fce9-6cb3-eba3-015a0b106163(a)gmail.com> > >> Content-Type: text/plain; charset="utf-8"; Format="flowed" > >> > >> > >> > >> On 10/16/2017 02:27 PM, Ismael Lemhadri wrote: > >> > @Andreas Muller: > >> > My references do not assume centering, e.g. > >> > http://ufldl.stanford.edu/wiki/index.php/PCA > >> > any reference? > >> > > >> It kinda does but is not very clear about it: > >> > >> This data has already been pre-processed so that each of the > >> features\textstyle x_1and\textstyle x_2have about the same mean (zero) > >> and variance. > >> > >> > >> > >> Wikipedia is much clearer: > >> Consider a datamatrix > >> <https://en.wikipedia.org/wiki/Matrix_%28mathematics%29>,*X*, with > >> column-wise zeroempirical mean > >> <https://en.wikipedia.org/wiki/Empirical_mean>(the sample mean of each > >> column has been shifted to zero), where each of the/n/rows represents a > >> different repetition of the experiment, and each of the/p/columns gives > >> a particular kind of feature (say, the results from a particular > sensor). > >> https://en.wikipedia.org/wiki/Principal_component_analysis#Details > >> > >> I'm a bit surprised to find that ESL says "The SVD of the centered > >> matrix X is another way of expressing the principal components of the > >> variables in X", > >> so they assume scaling? They don't really have a great treatment of PCA, > >> though. > >> > >> Bishop <http://www.springer.com/us/book/9780387310732> and Murphy > >> <https://mitpress.mit.edu/books/machine-learning-0> are pretty clear > >> that they subtract the mean (or assume zero mean) but don't standardize. > >>

3 3

Can fit a model with a target array of probabilities?
by Stuart Reynolds Oct. 18, 2017

Oct. 18, 2017

I'd like to fit a model that maps a matrix of continuous inputs to a target that's between 0 and 1 (a probability). In principle, I'd expect logistic regression should work out of the box with no modification (although its often posed as being strictly for classification, its loss function allows for fitting targets in the range 0 to 1, and not strictly zero or one.) However, scikit's LogisticRegression and LogisticRegressionCV reject target arrays that are continuous. Other LR implementations allow a matrix of probability estimates. Looking at: http://scikit-learn-general.narkive.com/4dSCktaM/using-logistic-regression-… and the fix here: https://github.com/scikit-learn/scikit-learn/pull/5084, which disables continuous inputs, it looks like there was some reason for this. So ... I'm looking for alternatives. SGDClassifier allows log loss and (if I understood the docs correctly) adds a logistic link function, but also rejects continuous targets. Oddly, SGDRegressor only allows ‘squared_loss’, ‘huber’, ‘epsilon_insensitive’, or ‘squared_epsilon_insensitive’, and doesn't seems to give a logistic function. In principle, GLM allow this, but scikit's docs say the GLM models only allows strict linear functions of their input, and doesn't allow a logistic link function. The docs direct people to the LogisticRegression class for this case. In R, there is: glm(Total_Service_Points_Won/Total_Service_Points_Played ~ ... , family = binomial(link=logit), weights = Total_Service_Points_Played) which would be ideal. Is something similar available in scikit? (Or any continuous model that takes and 0 to 1 target and outputs a 0 to 1 target?) I was surprised to see that the implementation of CalibratedClassifierCV(method="sigmoid") uses an internal implementation of logistic regression to do its logistic regressing -- which I can use, although I'd prefer to use a user-facing library. Thanks, - Stuart

4 14

Unclear help file about sklearn.decomposition.pca
by Ismael Lemhadri Oct. 17, 2017

Oct. 17, 2017

Thank you all for your feedback. The initial problem I came with wasnt the definition of PCA but what the sklearn method does. In practice I would always make sure the data is both centered and scaled before performing PCA. This is the recommended method because without scaling, the biggest direction could wrongly seem to explain a huge fraction of the variance. So my point was simply to clarify in the help file and the user guide what the PCA class does precisely to leave no unclarity to the reader. Moving forward I have now submitted a pull request on github as initially suggested by Roman on this thread. Best, Ismael On Mon, 16 Oct 2017 at 11:49 AM, <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: 1. Re: unclear help file for sklearn.decomposition.pca > (Andreas Mueller) > 2. Re: 1. Re: unclear help file for sklearn.decomposition.pca > (Oliver Tomic) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Mon, 16 Oct 2017 14:44:51 -0400 > From: Andreas Mueller <t3kcit(a)gmail.com> > To: scikit-learn(a)python.org > Subject: Re: [scikit-learn] 1. Re: unclear help file for > sklearn.decomposition.pca > Message-ID: <35142868-fce9-6cb3-eba3-015a0b106163(a)gmail.com> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > > > On 10/16/2017 02:27 PM, Ismael Lemhadri wrote: > > @Andreas Muller: > > My references do not assume centering, e.g. > > http://ufldl.stanford.edu/wiki/index.php/PCA > > any reference? > > > It kinda does but is not very clear about it: > > This data has already been pre-processed so that each of the > features\textstyle x_1and\textstyle x_2have about the same mean (zero) > and variance. > > > > Wikipedia is much clearer: > Consider a datamatrix > <https://en.wikipedia.org/wiki/Matrix_%28mathematics%29>,*X*, with > column-wise zeroempirical mean > <https://en.wikipedia.org/wiki/Empirical_mean>(the sample mean of each > column has been shifted to zero), where each of the/n/rows represents a > different repetition of the experiment, and each of the/p/columns gives > a particular kind of feature (say, the results from a particular sensor). > https://en.wikipedia.org/wiki/Principal_component_analysis#Details > > I'm a bit surprised to find that ESL says "The SVD of the centered > matrix X is another way of expressing the principal components of the > variables in X", > so they assume scaling? They don't really have a great treatment of PCA, > though. > > Bishop <http://www.springer.com/us/book/9780387310732> and Murphy > <https://mitpress.mit.edu/books/machine-learning-0> are pretty clear > that they subtract the mean (or assume zero mean) but don't standardize. >

3 2