Mailman 3 June 2016 - scikit-learn

partial_fit implementation for IsolationForest
by donkey-hotei＠cryptolab.net July 1, 2016

July 1, 2016

hello scikit-learn devs, After following the work on IsolationForest so far and testing on a real-world problem here we've found this model to be very promising for anomaly detection. However, at present, IsolationForest only fits data in batch even while it may be well suited to incremental on-line learning since one could subsample recent history and older estimators can be dropped progressively. I'd like to contribute this feature, but being new to ML and scikit-learn I'm curious how I should start making a quick & dirty version to see how this may work. Are there other good examples where one could see the difference between .fit and .partial_fit in other models? thanks isaak y.

6 9

Adding BM25 to sklearn.feature_extraction.text (Update)
by Basil Beirouti June 30, 2016

June 30, 2016

Hello everyone, I have successfully created a few versions of the BM25Transformer. I looked at TFIDFTransformer for guidance and I noticed that it outputs a sparse matrix when given a sparse termcount matrix as an input. Unfortunately, the fastest implementation of BM25Transformer that I have been able to come up with does NOT output a sparse matrix, it will return a regular numpy matrix. Benchmarked against the entire 20newsgroups corpus, here is how they perform (assuming input is csr_matrix for all): 1.) finishes in 4 seconds, outputs a regular numpy matrix 2.) finishes in 30 seconds, outputs a dok_matrix 3.) finishes in 130 seconds, outputs a regular numpy matrix It's worth noting that using algorithm 1 and converting the output to a sparse matrix still takes less time than 3, and takes about as long as 2. So my question is, how important is it that my BM25Transformer outputs a sparse matrix? I'm going to try another implementation which looks directly at the data, indices, and indptr attributes of the inputted csr_matrix. I just wanted to check in and see what people thought. Sincerely, Basil Beirouti

3 2

Re: [scikit-learn] [Scikit-learn-general] Gradient Descent
by Daniel Sullivan June 29, 2016

June 29, 2016

(Sent to wrong mailing list, sorry for duplication) Hi Chaitanya, Yes, Stochastic Gradient Descents algorithm logic is written in Cython. The implementation can be viewed here: https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/linear_mod… Hope that helps, Danny On Wed, Jun 29, 2016 at 10:31 AM, Chaitanya Prasad <chaitanya94(a)gmail.com> wrote: > Hello > > I'm a student currently trying to benchmark a few Black box optimized > algorithms against Gradient Descent Algorithms. It would be extremely > helpful for me if someone could tell me whether the Stochastic Gradient > Descent for scikit has been written in pure Python or whether it has been > optmized using Cython in any manner. > > Thanks and Regards. > Chaitanya > > > ------------------------------------------------------------------------------ > Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San > Francisco, CA to explore cutting-edge tech and listen to tech luminaries > present their vision of the future. This family event has something for > everyone, including kids. Get more information and register today. > http://sdm.link/attshape > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > >

1 0

How do we define a distance metric's parameter for grid search
by Hugo Ferreira June 28, 2016

June 28, 2016

Hello, I have posted this question in Stackoverflow and did not get an answer. This seems to be a basic usage question and am therefore sending it here. I have following code snippet that attempts to do a grid search in which one of the grid parameters are the distance metrics to be used for the KNN algorithm. The example below fails if I use "wminkowski", "seuclidean" or "mahalanobis" distances metrics. # Define the parameter values that should be searched k_range = range(1,31) weights = ['uniform' , 'distance'] algos = ['auto', 'ball_tree', 'kd_tree', 'brute'] leaf_sizes = range(10, 60, 10) metrics = ["euclidean", "manhattan", "chebyshev", "minkowski", "mahalanobis"] param_grid = dict(n_neighbors = list(k_range), weights = weights, algorithm = algos, leaf_size = list(leaf_sizes), metric=metrics) param_grid # Instantiate the algorithm knn = KNeighborsClassifier(n_neighbors=10) # Instantiate the grid grid = GridSearchCV(knn, param_grid=param_grid, cv=10, scoring='accuracy', n_jobs=-1) # Fit the models using the grid parameters grid.fit(X,y) I assume this is because I have to set or define the ranges for the various distance parameters (for example p, w for “wminkowski” - WMinkowskiDistance ). The "minkowski" distance may be working because its "p" parameter has the default 2. So my questions are: 1. Can we set the range of parameters for the distance metrics for the grid search and if so how? 2. Can we set the value of a parameters for the distance metrics for the grid search and if so how? Hope the question is clear. TIA

3 7

Spherical Kmeans #OT
by JAGANADH G June 28, 2016

June 28, 2016

Hi , is there any Python package available for experiment with Sperical Kmeans ? -- ********************************** JAGANADH G http://jaganadhg.in *ILUGCBE* http://ilugcbe.org.in

4 7

Welcome Loic Esteve (@lesteve) as a new core contributor
by Gael Varoquaux June 23, 2016

June 23, 2016

Hi, I'd like to welcome Loic Esteve (@lesteve) as a new core contributor to the scikit-learn team. Loic has been reviewing very seriously a number of PR, beyond his own contributions. It's great to have him on board! Cheers, Gaël

5 4

New Contributor
by Sagar Kar June 23, 2016

June 23, 2016

Hi Developers, I am new to scikit-learn and want some guidance to start contributing to it. I am well versed with python and c/c++. Linux user. Can anyone help me get me going. Regards, *Sagar Kar* P: +91-9085986905 W: sagarkar10.github.io

8 10

Random forest fitting very well
by muhammad waseem June 23, 2016

June 23, 2016

Hi All, I am trying to use random forests for a regression problem, with 10 input variables and one output variable. I am getting very good fit even with default parameters and low n_estimators. Even with n_estimator = 10, I get R^2 value of 0.95 on testing dataset (MSE=23) and a value of 0.99 for the training set. I was wondering, if this is common with random forest or I am missing something, Could you please share your experience? The total number of sample (training +testing) are equal to 10971. Also, what are the most important parameters (max_depth, bootstrap, max_leaf_nodes etc.) that I need to play with to tune my model even further? Lastly, is there is a way I can visualise a single tree of my forest (just for demonstration purposes)? Please see a figure below to demonstrate how well it is fitting with default values. [image: Inline image 1] Thanks Kindest Regards Waseem

3 4

Re: [scikit-learn] Decoding Differences Between SKL SVM and Matlab Libsvm Even When Parameters the Same
by Taylor, Johnmark June 22, 2016

June 22, 2016

Many thanks for the responses thus far! *Did you fix the random seeds across implementations as well? Differencesin seeds or generators might explain this.* The implementation of libsvm used by Matlab always has a seed of 1. I tried setting the seed for SKL SVM to 1 (and 0, 2, 3, and 4) as well, and the results were still different. *Did you try using the Python API to libsvm directly instead of through SKL?I'm guessing you have it on your computer since you have the Matlab API.That would at least let you test whether it's the fake data or whether it'sSKL.* I'll give that a shot next, thanks! *Also are you loading the fake data from a .mat file into Python (e.g. withthe SciPy 'loadmat' function) or are you generating it from a script? Maybesome weird floating point error between Python and Matlab is giving you thedifferent results? This could happen if you generate the data with a scriptwritten in both Python and Matlab, for example... along the same lines asthe random seed generator giving different results* I'm generating the fake data with a Python script and saving it to a .txt file, which is then loaded in by Python and Matlab in their respective scripts. To make sure there's no truncation error going on when they load in this .txt file to get the fake data, I applied the floor function to both sets of vectors (to make them ints) in both the Python and Matlab scripts, and they still give different results. So I don't think it's a data issue.

2 2

Re: [scikit-learn] scikit-learn Digest, Vol 3, Issue 37
by Jonathan Taylor June 22, 2016

June 22, 2016

import numpy as np import nose.tools as nt from sklearn.isotonic import isotonic_regression def test_isotonic_ymin_ymax(): X = np.array([1.26, 1.31,-0.57, 0.30, -0.70, -0.17, -1.59, 1.05, 1.39, 1.90, 0.20, 0.03, -0.08, 0.44, 0.01, -0.37, -0.89, -0.37, -1.32, 0.18]) X_iso = isotonic_regression(X, y_min=0., y_max=0.1) nt.assert_true(np.all((X_iso <= 0.1) * (X_iso >= 0.))) test_isotonic_ymin_ymax() On Wed, Jun 22, 2016 at 12:27 AM, <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. isotonic regression weird behavior? (Jonathan Taylor) > 2. Re: isotonic regression weird behavior? (Jonathan Taylor) > 3. Re: isotonic regression weird behavior? (Jonathan Taylor) > 4. Re: isotonic regression weird behavior? (Gael Varoquaux) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 21 Jun 2016 19:18:12 -0700 > From: Jonathan Taylor <jonathan.taylor(a)stanford.edu> > To: scikit-learn(a)python.org > Subject: [scikit-learn] isotonic regression weird behavior? > Message-ID: > < > CANmCCuSkekfbjH0d450_Cm5V3JYVXsSgEyGtj3pNmA6Ek8CssQ(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Was trying to fit isotonic regression with non-trivial y_min and y_max: > > In [*17*]: X > > Out[*17*]: > > array([ 1.26336413, 1.31853693, -0.57200917, 0.3072928 , -0.70686507, > > -0.17614937, -1.59943059, 1.05908504, 1.3958263 , 1.90580318, > > 0.20992272, 0.02836316, -0.08092235, 0.44438247, 0.01791253, > > -0.3771914 , -0.89577538, -0.37726249, -1.32687569, 0.18013201]) > > > In [*18*]: iso.isotonic_regression(X, y_min=0, y_max=0.1) > > Out[*18*]: > > array([-0.00826919, -0.00826919, -0.00826919, -0.00826919, -0.00826919, > > -0.00826919, -0.00826919, 0.10449344, 0.10449344, 0.10449344, > > 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344, > > 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344]) > > > The solution does not satisfy the bounds that each entry should be in > [0,0.1] > > > > -- > Jonathan Taylor > Dept. of Statistics > Sequoia Hall, 137 > 390 Serra Mall > Stanford, CA 94305 > Tel: 650.723.9230 > Fax: 650.725.8977 > Web: http://www-stat.stanford.edu/~jtaylo >

1 0