Mailman 3 August 2016 - scikit-learn

Gradient Boosting: Feature Importances do not sum to 1
by Douglas Chan Sept. 13, 2016

Sept. 13, 2016

Hello everyone, I notice conditions when Feature Importance values do not add up to 1 in ensemble tree methods, like Gradient Boosting Trees or AdaBoost Trees. I wonder if there’s a bug in the code. This error occurs when the ensemble has a large number of estimators. The exact conditions depend variously. For example, the error shows up sooner with a smaller amount of training samples. Or, if the depth of the tree is large. When this error appears, the predicted value seems to have converged. But it’s unclear if the error is causing the predicted value not to change with more estimators. In fact, the feature importance sum goes lower and lower with more estimators thereafter. I wonder if we’re hitting some floating point calculation error. Looking forward to hear your thoughts on this. Thank you! -Doug

3 5

Declaring numpy and scipy dependencies?
by Andreas Mueller Sept. 1, 2016

Sept. 1, 2016

Hi all. I think since the pipy ecosystem improved a lot, we should properly declare the scipy and numpy dependencies, so that ``pip install scikit-learn`` works properly. The argument why we did not do this previously was that this would try to download and build numpy and scipy, which are pretty much guaranteed to fail / result in a very slow installation. This is no longer true. Any opinions? Andy

5 10

MLPClassifier release
by Gartland, Myles Aug. 31, 2016

Aug. 31, 2016

Curious when .18 will be released. I am specifically interested in the MLPClassifier to show my students in class. Not sure I want them on the dev fork just yet.

2 1

How to deal with minor inconsistencies in scikit source code?
by Ilya Persky Aug. 30, 2016

Aug. 30, 2016

Hi All! I'm now reading scikit-learn source code and sometimes meet minor inconsistencies here and there like unnecessary copying of some array or some very unimportant race condition. Nothing like serious bug really. What should I do about it? Create an issue for each case would be an overkill. Create an issue for all of them and add pull request with fixes? Or first send a letter with them here?.. Again I'm new to this code and can be easily missing something (something looking like minor bug could appear to be a feature :) ). -- Thank you, Ilya.

2 1

Re: [scikit-learn] [Scikit-learn-general] Dropping Python 2.6 compatibility
by Andy Aug. 30, 2016

Aug. 30, 2016

Hi all. Picking up this old thread, I propose we announce that 0.18 is the last release that will support Python 2.6. That will give people some time to think about it between releases. Wdyt? Andy On 01/04/2016 08:28 AM, Gael Varoquaux wrote: > Happy new year everybody, > > As a new year resolution, I suggest that we drop Python 2.6 > compatibility. > > For an argumentation in this favor, see > http://www.snarky.ca/stop-using-python-2-6 (I don't buy everything there, > but the core idea is there). > > For us, this will mean more usage of context managers, which is good. > > The down side is that many clusters run RedHat variant that are still > under 2.6 (Duh!). The question is: are people using the stock Python on > the clusters, or something else. > > Opinions please? > > Gaël > > ------------------------------------------------------------------------------ > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general

2 1

Issue with DecisionTreeClassifier
by Ibrahim Dalal Aug. 30, 2016

Aug. 30, 2016

Dear Developers, DecisionTreeClassifier.decision_path() as used here http://scikit-learn.org/dev/auto_examples/tree/unveil_tree_structure.html is giving the following error: AttributeError: 'DecisionTreeClassifier' object has no attribute 'decision_path' Kindly help. Thanks

5 10

Re: [scikit-learn] Does NMF optimise over observed values
by Arthur Mensch Aug. 29, 2016

Aug. 29, 2016

Zeros are considered as zeros in the objective function, not as missing values - - i.e. no mask in the loss function. Le 28 août 2016 16:58, "Raphael C" <drraph(a)gmail.com> a écrit : What I meant was, how is the objective function defined when X is sparse? Raphael On Sunday, August 28, 2016, Raphael C <drraph(a)gmail.com> wrote: > Reading the docs for http://scikit-learn.org/st > able/modules/generated/sklearn.decomposition.NMF.html it says > > The objective function is: > > 0.5 * ||X - WH||_Fro^2 > + alpha * l1_ratio * ||vec(W)||_1 > + alpha * l1_ratio * ||vec(H)||_1 > + 0.5 * alpha * (1 - l1_ratio) * ||W||_Fro^2 > + 0.5 * alpha * (1 - l1_ratio) * ||H||_Fro^2 > > Where: > > ||A||_Fro^2 = \sum_{i,j} A_{ij}^2 (Frobenius norm) > ||vec(A)||_1 = \sum_{i,j} abs(A_{ij}) (Elementwise L1 norm) > > This seems to suggest that it is optimising over all values in X even if X is sparse. When using NMF for collaborative filtering we need the objective function to be defined over only the defined elements of X. The remaining elements should effectively be regarded as missing. > > > What is the true objective function NMF is using? > > > Raphael > > _______________________________________________ scikit-learn mailing list scikit-learn(a)python.org https://mail.python.org/mailman/listinfo/scikit-learn

5 7

Latent Semantic Analysis (LSA) and TrucatedSVD
by Roman Yurchak Aug. 29, 2016

Aug. 29, 2016

Hi all, I have a question about using the TruncatedSVD method for performing Latent Semantic Analysis/Indexing (LSA/LSI). The docs imply that simply applying TruncatedSVD to a tf-idf matrice is sufficient (cf. http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.Trun…), but I'm wondering about that. As far as I understood for LSA one computes a truncated SVD decomposition of the tf-idf matrix X (n_features x n_samples), X ≈ U @ Sigma @ V.T and then for a document vector d, the projection is computed as, d_proj = d.T @ U @ Sigma⁻¹ (source: http://nlp.stanford.edu/IR-book/pdf/18lsi.pdf) However, TruncatedSVD.fit_transform only computes, d_proj = d.T @ U and what's more does not store the singular values (Sigma) internally, so it cannot be easily applied afterwards. (the above notation are transposed with respect to those in the scikit learn docs). For instance, I have tried reproducing LSA decomposition from literature and I'm not getting the expected results unless I perform an additional normalization by the Sigma matrix: https://gist.github.com/rth/3af30c60bece7db4207821a6dddc5e8d I was wondering if I am missing something here? Thank you, -- Roman

4 5

Fwd: inconsistency between libsvm and scikit-learn.svc results
by elgesto＠gmail.com Aug. 28, 2016

Aug. 28, 2016

I have a project that is based on SVM algorithm implemented by libsvm <https://www.csie.ntu.edu.tw/~cjlin/libsvm/>. Recently I decided to try several other classification algorithm, this is where scikit-learn <http://scikit-learn.org/> comes to the picture. The connection to the scikit was pretty straightforward, it supports libsvm format by load_svmlight_file routine. Ans it's svm implementation is based on the same libsvm. When everything was done, I decided to the check the consistence of the results by directly running libsvm and via scikit-learn, and the results were different. Among 18 measures in learning curves, 7 were different, and the difference is located at the small steps of the learning curve. The libsvm results seems much more stable, but scikit-learn results have some drastic fluctuation. The classifiers have exactly the same parameters of course. I tried to check the version of libsvm in scikit-learn implementation, but I din't find it, the only thing I found was libsvm.so file. Currently I am using libsvm 3.21 version, and scikit-learn 0.17.1 version. I wound appreciate any help in addressing this issue. size libsvm scikit-learn 1 0.1336239435355727 0.1336239435355727 2 0.08699516468193455 0.08699516468193455 3 0.32928301642777424 0.2117238289550198 #different 4 0.2835688734876902 0.2835688734876902 5 0.27846766962743097 0.26651875338163966 #different 6 0.2853854654662907 0.18898048915599963 #different 7 0.28196058132165136 0.28196058132165136 8 0.31473956032575623 0.1958710201604552 #different 9 0.33588303670653136 0.2101641630182972 #different 10 0.4075242509025311 0.2997807499800962 #different 15 0.4391771087975972 0.4391771087975972 20 0.3837789445609818 0.2713167833345173 #different 25 0.4252154334940311 0.4252154334940311 30 0.4256407777477492 0.4256407777477492 35 0.45314944605858387 0.45314944605858387 40 0.4278633233755064 0.4278633233755064 45 0.46174762022239796 0.46174762022239796 50 0.45370452524846866 0.45370452524846866

6 8

Does NMF optimise over observed values
by Raphael C Aug. 28, 2016

Aug. 28, 2016

Reading the docs for http://scikit-learn.org/stable/modules/generated/sklearn.decomposition.NMF.… it says The objective function is: 0.5 * ||X - WH||_Fro^2 + alpha * l1_ratio * ||vec(W)||_1 + alpha * l1_ratio * ||vec(H)||_1 + 0.5 * alpha * (1 - l1_ratio) * ||W||_Fro^2 + 0.5 * alpha * (1 - l1_ratio) * ||H||_Fro^2 Where: ||A||_Fro^2 = \sum_{i,j} A_{ij}^2 (Frobenius norm) ||vec(A)||_1 = \sum_{i,j} abs(A_{ij}) (Elementwise L1 norm) This seems to suggest that it is optimising over all values in X even if X is sparse. When using NMF for collaborative filtering we need the objective function to be defined over only the defined elements of X. The remaining elements should effectively be regarded as missing. What is the true objective function NMF is using? Raphael

1 1