Mailman 3 June 2016 - scikit-learn

partial_fit implementation for IsolationForest
by donkey-hotei＠cryptolab.net July 1, 2016

July 1, 2016

hello scikit-learn devs, After following the work on IsolationForest so far and testing on a real-world problem here we've found this model to be very promising for anomaly detection. However, at present, IsolationForest only fits data in batch even while it may be well suited to incremental on-line learning since one could subsample recent history and older estimators can be dropped progressively. I'd like to contribute this feature, but being new to ML and scikit-learn I'm curious how I should start making a quick & dirty version to see how this may work. Are there other good examples where one could see the difference between .fit and .partial_fit in other models? thanks isaak y.

6 9

Adding BM25 to sklearn.feature_extraction.text (Update)
by Basil Beirouti June 30, 2016

June 30, 2016

Hello everyone, I have successfully created a few versions of the BM25Transformer. I looked at TFIDFTransformer for guidance and I noticed that it outputs a sparse matrix when given a sparse termcount matrix as an input. Unfortunately, the fastest implementation of BM25Transformer that I have been able to come up with does NOT output a sparse matrix, it will return a regular numpy matrix. Benchmarked against the entire 20newsgroups corpus, here is how they perform (assuming input is csr_matrix for all): 1.) finishes in 4 seconds, outputs a regular numpy matrix 2.) finishes in 30 seconds, outputs a dok_matrix 3.) finishes in 130 seconds, outputs a regular numpy matrix It's worth noting that using algorithm 1 and converting the output to a sparse matrix still takes less time than 3, and takes about as long as 2. So my question is, how important is it that my BM25Transformer outputs a sparse matrix? I'm going to try another implementation which looks directly at the data, indices, and indptr attributes of the inputted csr_matrix. I just wanted to check in and see what people thought. Sincerely, Basil Beirouti

3 2

Re: [scikit-learn] [Scikit-learn-general] Gradient Descent
by Daniel Sullivan June 29, 2016

June 29, 2016

(Sent to wrong mailing list, sorry for duplication) Hi Chaitanya, Yes, Stochastic Gradient Descents algorithm logic is written in Cython. The implementation can be viewed here: https://github.com/scikit-learn/scikit-learn/blob/master/sklearn/linear_mod… Hope that helps, Danny On Wed, Jun 29, 2016 at 10:31 AM, Chaitanya Prasad <chaitanya94(a)gmail.com> wrote: > Hello > > I'm a student currently trying to benchmark a few Black box optimized > algorithms against Gradient Descent Algorithms. It would be extremely > helpful for me if someone could tell me whether the Stochastic Gradient > Descent for scikit has been written in pure Python or whether it has been > optmized using Cython in any manner. > > Thanks and Regards. > Chaitanya > > > ------------------------------------------------------------------------------ > Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San > Francisco, CA to explore cutting-edge tech and listen to tech luminaries > present their vision of the future. This family event has something for > everyone, including kids. Get more information and register today. > http://sdm.link/attshape > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > >

1 0

How do we define a distance metric's parameter for grid search
by Hugo Ferreira June 28, 2016

June 28, 2016

Hello, I have posted this question in Stackoverflow and did not get an answer. This seems to be a basic usage question and am therefore sending it here. I have following code snippet that attempts to do a grid search in which one of the grid parameters are the distance metrics to be used for the KNN algorithm. The example below fails if I use "wminkowski", "seuclidean" or "mahalanobis" distances metrics. # Define the parameter values that should be searched k_range = range(1,31) weights = ['uniform' , 'distance'] algos = ['auto', 'ball_tree', 'kd_tree', 'brute'] leaf_sizes = range(10, 60, 10) metrics = ["euclidean", "manhattan", "chebyshev", "minkowski", "mahalanobis"] param_grid = dict(n_neighbors = list(k_range), weights = weights, algorithm = algos, leaf_size = list(leaf_sizes), metric=metrics) param_grid # Instantiate the algorithm knn = KNeighborsClassifier(n_neighbors=10) # Instantiate the grid grid = GridSearchCV(knn, param_grid=param_grid, cv=10, scoring='accuracy', n_jobs=-1) # Fit the models using the grid parameters grid.fit(X,y) I assume this is because I have to set or define the ranges for the various distance parameters (for example p, w for “wminkowski” - WMinkowskiDistance ). The "minkowski" distance may be working because its "p" parameter has the default 2. So my questions are: 1. Can we set the range of parameters for the distance metrics for the grid search and if so how? 2. Can we set the value of a parameters for the distance metrics for the grid search and if so how? Hope the question is clear. TIA

3 7

Spherical Kmeans #OT
by JAGANADH G June 28, 2016

June 28, 2016

Hi , is there any Python package available for experiment with Sperical Kmeans ? -- ********************************** JAGANADH G http://jaganadhg.in *ILUGCBE* http://ilugcbe.org.in

4 7

Welcome Loic Esteve (@lesteve) as a new core contributor
by Gael Varoquaux June 23, 2016

June 23, 2016

Hi, I'd like to welcome Loic Esteve (@lesteve) as a new core contributor to the scikit-learn team. Loic has been reviewing very seriously a number of PR, beyond his own contributions. It's great to have him on board! Cheers, Gaël

5 4

New Contributor
by Sagar Kar June 23, 2016

June 23, 2016

Hi Developers, I am new to scikit-learn and want some guidance to start contributing to it. I am well versed with python and c/c++. Linux user. Can anyone help me get me going. Regards, *Sagar Kar* P: +91-9085986905 W: sagarkar10.github.io

8 10

Random forest fitting very well
by muhammad waseem June 23, 2016

June 23, 2016

Hi All, I am trying to use random forests for a regression problem, with 10 input variables and one output variable. I am getting very good fit even with default parameters and low n_estimators. Even with n_estimator = 10, I get R^2 value of 0.95 on testing dataset (MSE=23) and a value of 0.99 for the training set. I was wondering, if this is common with random forest or I am missing something, Could you please share your experience? The total number of sample (training +testing) are equal to 10971. Also, what are the most important parameters (max_depth, bootstrap, max_leaf_nodes etc.) that I need to play with to tune my model even further? Lastly, is there is a way I can visualise a single tree of my forest (just for demonstration purposes)? Please see a figure below to demonstrate how well it is fitting with default values. [image: Inline image 1] Thanks Kindest Regards Waseem

3 4

Re: [scikit-learn] Decoding Differences Between SKL SVM and Matlab Libsvm Even When Parameters the Same
by Taylor, Johnmark June 22, 2016

June 22, 2016

Many thanks for the responses thus far! *Did you fix the random seeds across implementations as well? Differencesin seeds or generators might explain this.* The implementation of libsvm used by Matlab always has a seed of 1. I tried setting the seed for SKL SVM to 1 (and 0, 2, 3, and 4) as well, and the results were still different. *Did you try using the Python API to libsvm directly instead of through SKL?I'm guessing you have it on your computer since you have the Matlab API.That would at least let you test whether it's the fake data or whether it'sSKL.* I'll give that a shot next, thanks! *Also are you loading the fake data from a .mat file into Python (e.g. withthe SciPy 'loadmat' function) or are you generating it from a script? Maybesome weird floating point error between Python and Matlab is giving you thedifferent results? This could happen if you generate the data with a scriptwritten in both Python and Matlab, for example... along the same lines asthe random seed generator giving different results* I'm generating the fake data with a Python script and saving it to a .txt file, which is then loaded in by Python and Matlab in their respective scripts. To make sure there's no truncation error going on when they load in this .txt file to get the fake data, I applied the floor function to both sets of vectors (to make them ints) in both the Python and Matlab scripts, and they still give different results. So I don't think it's a data issue.

2 2

Re: [scikit-learn] scikit-learn Digest, Vol 3, Issue 37
by Jonathan Taylor June 22, 2016

June 22, 2016

import numpy as np import nose.tools as nt from sklearn.isotonic import isotonic_regression def test_isotonic_ymin_ymax(): X = np.array([1.26, 1.31,-0.57, 0.30, -0.70, -0.17, -1.59, 1.05, 1.39, 1.90, 0.20, 0.03, -0.08, 0.44, 0.01, -0.37, -0.89, -0.37, -1.32, 0.18]) X_iso = isotonic_regression(X, y_min=0., y_max=0.1) nt.assert_true(np.all((X_iso <= 0.1) * (X_iso >= 0.))) test_isotonic_ymin_ymax() On Wed, Jun 22, 2016 at 12:27 AM, <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. isotonic regression weird behavior? (Jonathan Taylor) > 2. Re: isotonic regression weird behavior? (Jonathan Taylor) > 3. Re: isotonic regression weird behavior? (Jonathan Taylor) > 4. Re: isotonic regression weird behavior? (Gael Varoquaux) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 21 Jun 2016 19:18:12 -0700 > From: Jonathan Taylor <jonathan.taylor(a)stanford.edu> > To: scikit-learn(a)python.org > Subject: [scikit-learn] isotonic regression weird behavior? > Message-ID: > < > CANmCCuSkekfbjH0d450_Cm5V3JYVXsSgEyGtj3pNmA6Ek8CssQ(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Was trying to fit isotonic regression with non-trivial y_min and y_max: > > In [*17*]: X > > Out[*17*]: > > array([ 1.26336413, 1.31853693, -0.57200917, 0.3072928 , -0.70686507, > > -0.17614937, -1.59943059, 1.05908504, 1.3958263 , 1.90580318, > > 0.20992272, 0.02836316, -0.08092235, 0.44438247, 0.01791253, > > -0.3771914 , -0.89577538, -0.37726249, -1.32687569, 0.18013201]) > > > In [*18*]: iso.isotonic_regression(X, y_min=0, y_max=0.1) > > Out[*18*]: > > array([-0.00826919, -0.00826919, -0.00826919, -0.00826919, -0.00826919, > > -0.00826919, -0.00826919, 0.10449344, 0.10449344, 0.10449344, > > 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344, > > 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344]) > > > The solution does not satisfy the bounds that each entry should be in > [0,0.1] > > > > -- > Jonathan Taylor > Dept. of Statistics > Sequoia Hall, 137 > 390 Serra Mall > Stanford, CA 94305 > Tel: 650.723.9230 > Fax: 650.725.8977 > Web: http://www-stat.stanford.edu/~jtaylo >

1 0

isotonic regression weird behavior?
by Jonathan Taylor June 22, 2016

June 22, 2016

Was trying to fit isotonic regression with non-trivial y_min and y_max: In [*17*]: X Out[*17*]: array([ 1.26336413, 1.31853693, -0.57200917, 0.3072928 , -0.70686507, -0.17614937, -1.59943059, 1.05908504, 1.3958263 , 1.90580318, 0.20992272, 0.02836316, -0.08092235, 0.44438247, 0.01791253, -0.3771914 , -0.89577538, -0.37726249, -1.32687569, 0.18013201]) In [*18*]: iso.isotonic_regression(X, y_min=0, y_max=0.1) Out[*18*]: array([-0.00826919, -0.00826919, -0.00826919, -0.00826919, -0.00826919, -0.00826919, -0.00826919, 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344, 0.10449344]) The solution does not satisfy the bounds that each entry should be in [0,0.1] -- Jonathan Taylor Dept. of Statistics Sequoia Hall, 137 390 Serra Mall Stanford, CA 94305 Tel: 650.723.9230 Fax: 650.725.8977 Web: http://www-stat.stanford.edu/~jtaylo

4 5

Decoding Differences Between SKL SVM and Matlab Libsvm Even When Parameters the Same
by Taylor, Johnmark June 22, 2016

June 22, 2016

Hello, I am moving much of my neuroimaging coding over to Python from Matlab and so I am switching from using libsvm in Matlab to using Scikit-learn SVM in Python. Just to make sure I am not changing anything substantive about my analyses, I am experimenting with the two implementations and trying to see whether I can get them to yield identical results. In Python I am using: clf = svm.SVC(kernel='linear',C=1,probability=True) In Matlab (libsvm) I am using: clf = libsvmtrain(svm_training_labels,svm_training_vectors,['-t 0 -b 1 -c 1']) When I run the SVM using these two different ways using simulated data, I get subtly different results, even though I have fixed all of the parameters of the SVMs to be the same using input arguments (linear classifier, C=1, use probability estimates), and even though all the other default parameters seem to be the same across these functions (tolerance = .001, both using shrinking heuristics by default). To give more details regarding the simulations: One simulation I ran was designed to be absurdly difficult--it yielded 40% accuracy for Matlab libsvm, and 44% accuracy for scikit-learn svm (binary classification, chance = 50%). In this simulation, the two SVMs agreed in their predictions only 18% of the time (in other words, they were both not only guessing below chance, but they nearly always gave opposite guesses compared to each other). The other simulation was easier, yielding 68% accuracy for Matlab libsvm, and 67% accuracy for scikit-learn SVM. In this simulation, the two SVMs agreed in their predictions 97% of the time. So even though they often got it wrong, they tended to make the same wrong guesses. Any idea of what could possibly be leading to differences in the results? My understanding is that SKL uses libsvm under the hood, so it's a been confusing why the decoders are behaving differently. Both analyses are being run on the same computer (Linux OS). Thank you very much, JohnMark Taylor PhD Student, Harvard Vision Sciences Lab

3 2

The sum of feature importances !=1
by Enhui HUANG June 21, 2016

June 21, 2016

Hi, When I ran the following code: X, y = make_classification(n_samples=100) clf = GradientBoostingClassifier(random_state=0).fit(X, y) imp=clf.feature_importances_ print "The sum of feature importances:", sum(imp) The sum of feature importances is not always equal to 1. So do you have a nice explanation for this situation? Besides, if a tree only contains a root, could we say all its feature importances are 0? I guess the root trees will influence sum of feature importances. Is it right? Best, Enhui

1 0

interpretation of manifold.MDS results
by Bill Ross June 19, 2016

June 19, 2016

I have a set of photos and a set of metrics for assigning distances between them. I am out to find the dimensionality of (space needed to embed) each set of distances, out of curiosity and to inform my use of the metrics in forming the character of an AI. It was suggested that manifold.MDS could help, and it looked perfect for a toy case I wrote. However, with actual data I found that the stress_ term doesn't go to 0 when the number of components reaches the number of points: http://stackoverflow.com/questions/37855596/calculate-the-spatial-dimension… I'm not sure if this question has achieved visibility, since I added the scikit-learn tag after it was created. I'd appreciate any discussion of the meaning of the value in the limit for stress_, and whether a meaningful dimensionality measure can be derived. Thanks, Bill

1 0

The culture of commit squashing
by Joel Nothman June 18, 2016

June 18, 2016

For the last few years, there's been a notion that we should squash PRs down to a single commit before merging. Squashing can give a cleaner commit history, and avoid overrepresentation of minor work given silly commit count metrics used by Github and others. I'm not sure if there are other motivations. Recently I've seen several contributors amending commits and force-pushing changes. I find this disruptive to the reviewing process in a number of ways (links are broken; what's changed is hard to discern, when it could have been indicated in a commit message and diff; etc.). I have had to ask these several users to cease and desist. I also find that performing the squash can be unnecessary overhead either for the merger or the PR developer. I think squashing is more trouble than it's worth, except where: * there are embarrassingly many minor commits in a PR * squashing first makes a rebase easier because of concurrent changes to the codebase * otherwise for cosmetic reasons only when there is low reviewing activity on the PR While squashing is far from the slowest part of our review process, being able to hit the merge button and move on would be great. Do others agree that a culture of amending commits in the ordinary case is counterproductive? (And apologies for wasting your time on such a silly issue, but I'm sick of clicking links in emails to find the commit's disappeared.)

10 15

Estimator.predict() thread safety
by Philip Tully June 17, 2016

June 17, 2016

Hi all, I notice when I train a model and expose the predict function through a web API, predict takes longer to run in a multi-threaded environment than a single-threaded one. I'm guessing the root cause has something to do with thread collisions but I must be doing something incorrectly within the code (I set n_jobs=-1 for both FeatureUnion and estimators/gridsearchers there) has someone else ran into a similar issue? I can provide more details if this Q is rather opaque still best, Philip

3 7

Question about error of LLE and backtransformation of coordinates
by Matthieu Brucher June 16, 2016

June 16, 2016

Hi! The errors are quite small compared to the machine precision. As the reduction is also an approximation of the underlying manifold, not an "isotropic" one as well (you can see int he example that red points are less squashed together than blue ones), you won't have a perfect reconstruction either. In a way, if you can reproduce in the reduced space the same distances (or barycenters for LLE) compared the original space, then you can have a perfect reconstruction (but it will still be subject to floating point precision). For the sphere, you can't: take 4 points, can you make the fourth as a barycenter of the other 3? No. That's the error you are seeing. Cheers, Matthieu 2016-06-16 10:12 GMT+01:00 Unger, Jörg <joerg.unger(a)bam.de>: > > I’ve tried the example that is available here > > > > http://scikit-learn.org/stable/auto_examples/manifold/plot_manifold_sphere.… > > > > These are essentially points on a 3D sphere, so the dimension of the embedded manifold is two. > > I’ve changed the example a little bit to extract the error as well. So instead of > > > > trans_data = manifold\ > > .LocallyLinearEmbedding(n_neighbors, 2, > > method=method).fit_transform(sphere_data).T > > > > I’ve done something like > > solver = manifold.LocallyLinearEmbedding(n_neighbors, dim_y, method=method) > > trans_data = solver.fit_transform(sphere_data).T > > error = solver.reconstruction_error_ > > > > I would have expected the error to be significant for dim_y=1, since I can’t reproduce with just a single coordinate the results. For dim_y=2, I expected a significant decrease, and for dim_y=3, I expected to exactly recover the original result. > > > > What I get is (for standard LLE) > > dim_y = 1 : error = 1.62031573333e-07 > > dim_y = 2 : error = 1.79465538543e-06 > > dim_y = 3 : error = 7.00280676182e-06 > > > > Could anyone explain, why I do not get the expected results? > > > > Furthermore, is there an option to retransform the coordinates from the local dimension to the global dimension? I’m interested in transforming the original global samples to local coordinates (this is done via the transform method), but then I would like to transform samples from coordinates in the embedded space back into the global space. > > > > Best regards, > > Jörg F. Unger > > > ------------------------------------------------------------------------------ > What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic > patterns at an interface-level. Reveals which users, apps, and protocols are > consuming the most bandwidth. Provides multi-vendor support for NetFlow, > J-Flow, sFlow and other flows. Make informed decisions using capacity planning > reports. http://pubads.g.doubleclick.net/gampad/clk?id=1444514421&iu=/41014381 > _______________________________________________ > Scikit-learn-general mailing list > Scikit-learn-general(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/scikit-learn-general > -- Information System Engineer, Ph.D. Blog: http://blog.audio-tk.com/ LinkedIn: http://www.linkedin.com/in/matthieubrucher -- Information System Engineer, Ph.D. Blog: http://blog.audio-tk.com/ LinkedIn: http://www.linkedin.com/in/matthieubrucher

1 0

Re: [scikit-learn] adding BM25 relevance function
by Basil Beirouti June 16, 2016

June 16, 2016

Hello Pavel and Joel, I forked the repository and cloned it on my machine. I'm using pycharm on a Mac, and while looking at text.py, I'm getting an unresolved reference for "xrange" at line 28: from ..externals.six.moves import range Pycharm says Function 'six.py' is too large to analyze, so I'm not sure if this error is somehow related to that. I decided to try to build the code as a sanity check but I can't find any reliable instructions as to how to do that. Naively, I opened terminal and cd to the directory above "scikit-learn" folder (where I had cloned my fork) and tried to run: $ python3 setup.py install Which didn't work. I got this error: ImportError: No module named 'sklearn' Can someone point me in the right direction? And how can the code try to import sklearn if it doesn't exist yet? Note I haven't installed the release version of scikit-learn using pip or any other tool, but I should be able to bootstrap it from the source code, right? Here's the full error message if it helps. Forgive me if it's a silly mistake, but I haven't found any reliable guidelines online. File "setup.py", line 84, in <module> from numpy.distutils.core import setup File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/distutils/core.py", line 26, in <module> from numpy.distutils.command import config, config_compiler, \ File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/distutils/command/build_ext.py", line 18, in <module> from numpy.distutils.system_info import combine_paths File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy/distutils/system_info.py", line 232, in <module> triplet = str(p.communicate()[0].decode().strip()) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 791, in communicate stdout = _eintr_retry_call(self.stdout.read) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 476, in _eintr_retry_call return func(*args) KeyboardInterrupt Basils-MacBook-Pro:sklearn basilbeirouti$ python3 setup.py install non-existing path in '__check_build': '_check_build.c' Appending sklearn.__check_build configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.__check_build') Appending sklearn._build_utils configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn._build_utils') Appending sklearn.covariance configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.covariance') Appending sklearn.covariance/tests configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.covariance/tests') Appending sklearn.cross_decomposition configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.cross_decomposition') Appending sklearn.cross_decomposition/tests configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.cross_decomposition/tests') Appending sklearn.feature_selection configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.feature_selection') Appending sklearn.feature_selection/tests configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.feature_selection/tests') Appending sklearn.gaussian_process configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.gaussian_process') Appending sklearn.gaussian_process/tests configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.gaussian_process/tests') Appending sklearn.mixture configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.mixture') Appending sklearn.mixture/tests configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.mixture/tests') Appending sklearn.model_selection configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.model_selection') Appending sklearn.model_selection/tests configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.model_selection/tests') Appending sklearn.neural_network configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.neural_network') Appending sklearn.neural_network/tests configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.neural_network/tests') Appending sklearn.preprocessing configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.preprocessing') Appending sklearn.preprocessing/tests configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.preprocessing/tests') Appending sklearn.semi_supervised configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.semi_supervised') Appending sklearn.semi_supervised/tests configuration to sklearn Ignoring attempt to set 'name' (from 'sklearn' to 'sklearn.semi_supervised/tests') Warning: Assuming default configuration (./_build_utils/{setup__build_utils,setup}.py was not found)Warning: Assuming default configuration (./covariance/{setup_covariance,setup}.py was not found)Warning: Assuming default configuration (./covariance/tests/setup_covariance/{setup_covariance/tests,setup}.py was not found)Warning: Assuming default configuration (./cross_decomposition/{setup_cross_decomposition,setup}.py was not found)Warning: Assuming default configuration (./cross_decomposition/tests/setup_cross_decomposition/{setup_cross_decomposition/tests,setup}.py was not found)Warning: Assuming default configuration (./feature_selection/{setup_feature_selection,setup}.py was not found)Warning: Assuming default configuration (./feature_selection/tests/setup_feature_selection/{setup_feature_selection/tests,setup}.py was not found)Warning: Assuming default configuration (./gaussian_process/{setup_gaussian_process,setup}.py was not found)Warning: Assuming default configuration (./gaussian_process/tests/setup_gaussian_process/{setup_gaussian_process/tests,setup}.py was not found)Warning: Assuming default configuration (./mixture/{setup_mixture,setup}.py was not found)Warning: Assuming default configuration (./mixture/tests/setup_mixture/{setup_mixture/tests,setup}.py was not found)Warning: Assuming default configuration (./model_selection/{setup_model_selection,setup}.py was not found)Warning: Assuming default configuration (./model_selection/tests/setup_model_selection/{setup_model_selection/tests,setup}.py was not found)Warning: Assuming default configuration (./neural_network/{setup_neural_network,setup}.py was not found)Warning: Assuming default configuration (./neural_network/tests/setup_neural_network/{setup_neural_network/tests,setup}.py was not found)Warning: Assuming default configuration (./preprocessing/{setup_preprocessing,setup}.py was not found)Warning: Assuming default configuration (./preprocessing/tests/setup_preprocessing/{setup_preprocessing/tests,setup}.py was not found)Warning: Assuming default configuration (./semi_supervised/{setup_semi_supervised,setup}.py was not found)Warning: Assuming default configuration (./semi_supervised/tests/setup_semi_supervised/{setup_semi_supervised/tests,setup}.py was not found)Traceback (most recent call last): File "setup.py", line 85, in <module> setup(**configuration(top_path='').todict()) File "setup.py", line 44, in configuration config.add_subpackage('cluster') File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/numpy/distutils/misc_util.py", line 1003, in add_subpackage caller_level = 2) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/numpy/distutils/misc_util.py", line 972, in get_subpackage caller_level = caller_level + 1) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/site-packages/numpy/distutils/misc_util.py", line 884, in _get_configuration_from_setup_py ('.py', 'U', 1)) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/imp.py", line 234, in load_module return load_source(name, filename, file) File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/imp.py", line 172, in load_source module = _load(spec) File "<frozen importlib._bootstrap>", line 693, in _load File "<frozen importlib._bootstrap>", line 673, in _load_unlocked File "<frozen importlib._bootstrap_external>", line 662, in exec_module File "<frozen importlib._bootstrap>", line 222, in _call_with_frames_removed File "./cluster/setup.py", line 8, in <module> from sklearn._build_utils import get_blas_info ImportError: No module named 'sklearn' On Tue, Jun 14, 2016 at 11:41 AM, <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: Adding BM25 relevance function (Pavel Soriano) > 2. Re: The culture of commit squashing (Andreas Mueller) > 3. Re: The culture of commit squashing (Tom DLT) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Tue, 14 Jun 2016 16:11:10 +0000 > From: Pavel Soriano <sorianopavel(a)gmail.com> > To: Scikit-learn user and developer mailing list > <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] Adding BM25 relevance function > Message-ID: > < > CAN0wWk93r2aw9No65CGiCW5hQG7-oFYVZaMJQpXpegTXMSqPLg(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hey, > > Good thing that you are trying to finish this. > > Well, I looked into my old notes, and the Delta tf-idf comes from the > "Delta > TFIDF: An Improved Feature Space for Sentiment Analysis" > <http://ebiquity.umbc.edu/_file_directory_/papers/446.pdf> paper. I guess > it is not very popular and apparently it has a drawback: it does not take > into account the number of times a word occurs in each document while > calculating the distribution amongst classes. At least that is what I wrote > on my notes... > > As for the delta idf... If it helps, I can look into my old code cause I do > not know what I was talking about. I guess it has to do somehow with the > paper cited before. > > Cheers, > > Pavel Soriano > > > > > On Tue, Jun 14, 2016 at 5:49 PM Basil Beirouti <basilbeirouti(a)gmail.com> > wrote: > > > Hi Joel, > > > > Thanks for your response and for digging up that archived thread, it > gives > > me a lot of clarity. > > > > I see your point about BM25, but I think in most cases where TFIDF makes > > sense, BM25 makes sense as well, but it could be "overkill". > > > > Consider that TFIDF does not produce normalized results either > > < > http://scikit-learn.org/stable/auto_examples/text/document_clustering.html#… > >, > > If BM25 requires dimensionality reduction (eg. using LSA) , so too would > > TFIDF. The term-document matrix is the same size no matter which > weighting > > scheme is used. The only difference is that BM25 produces better results > > when the corpus is large enough that the term frequency in a document, > and > > the document frequency in the corpus, can vary considerably across a > broad > > range of values.Maybe you could even say TFIDF and BM25 are the same > > equation except BM25 has a few additional hyperparameters (b and k). > > > > So is the advantage that BM25 provides for large diverse corpora with it? > > or is it marginal? Perhaps you can point me to some more examples where > > TFIDF is used (in supervised setting preferably) and I can plug in BM25 > in > > place of TFIDF and see how it compares. Here are some I found: > > > > > > > http://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_da… > > *(supervised)* > > > > > http://scikit-learn.org/stable/auto_examples/text/document_clustering.html#… > > (*unsupervised)* > > > > Thank you! > > Basil > > > > PS: By the way, I'm not familiar with the delta-idf transform that Pavel > > mentions in the archive you linked, I'll have to delve deeper into that. > I > > agree with the response to Pavel that he should be putting it in a > separate > > class, not adding on to the TFIDF. I think it would take me about 6-8 > weeks > > to adapt my code to the fit transform model and submit a pull request. > > > > > > > > > > > > > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > > > -- > Pavel SORIANO > > PhD Student > ERIC Laboratory > Universit? de Lyon >

3 2

Re: [scikit-learn] Adding BM25 relevance function
by Basil Beirouti June 14, 2016

June 14, 2016

Hi Joel, Thanks for your response and for digging up that archived thread, it gives me a lot of clarity. I see your point about BM25, but I think in most cases where TFIDF makes sense, BM25 makes sense as well, but it could be "overkill". Consider that TFIDF does not produce normalized results either <http://scikit-learn.org/stable/auto_examples/text/document_clustering.html#…>, If BM25 requires dimensionality reduction (eg. using LSA) , so too would TFIDF. The term-document matrix is the same size no matter which weighting scheme is used. The only difference is that BM25 produces better results when the corpus is large enough that the term frequency in a document, and the document frequency in the corpus, can vary considerably across a broad range of values.Maybe you could even say TFIDF and BM25 are the same equation except BM25 has a few additional hyperparameters (b and k). So is the advantage that BM25 provides for large diverse corpora with it? or is it marginal? Perhaps you can point me to some more examples where TFIDF is used (in supervised setting preferably) and I can plug in BM25 in place of TFIDF and see how it compares. Here are some I found: http://scikit-learn.org/stable/tutorial/text_analytics/working_with_text_da… *(supervised)* http://scikit-learn.org/stable/auto_examples/text/document_clustering.html#… (*unsupervised)* Thank you! Basil PS: By the way, I'm not familiar with the delta-idf transform that Pavel mentions in the archive you linked, I'll have to delve deeper into that. I agree with the response to Pavel that he should be putting it in a separate class, not adding on to the TFIDF. I think it would take me about 6-8 weeks to adapt my code to the fit transform model and submit a pull request.

2 1

Re: [scikit-learn] Adding BM25 relevance function to sklearn.feature_extraction.text
by Basil Beirouti June 14, 2016

June 14, 2016

Hello all, You can use sklearn.feature_extraction.text.TfidfVectorizer to learn a corpus of documents and rank them in order of relevance to a new previously unseen query. BM25 works in a similar manner to TfidfVectorizer, but is more complex and considered one of the most successful information retrieval algorithms. I currently have code that implements BM25 quite efficiently to learn a corpus of documents and I want to modify/port it to align with the fit-transform framework of sklearn. I think it could fit neatly into the current codebase. Questions: 1.) Would this be a desirable feature? 2.) Any advice for how to proceed with this? Things to watch out for? Any and all advice is welcome. Thanks! Basil

2 1

how to create and execute a machine learning models in Java/JVM based application (in production) using Python
by Gaurav gupta June 13, 2016

June 13, 2016

Hi All, Could you please guide me on how to *create and execute *a machine learning models/statistical models (regression, Decision tree, K means clustering, Naive bayes, scorecard/linear/logistic regression etc. and GBM, GLM ) in *Java/JVM based application* (in production). We have an ETL sort of Java based product where one can do most of data Preparation steps for machine learning, like data ingestion from JDBC, files, HDFS, No SQL etc., joins and aggregations etc.(which are required for Feature engineering) and now we want to add Analytics capabilities using machine learning/statistical modeling. Right now, we are using JPMML- evaluator <https://github.com/jpmml/jpmml-evaluator> to score the models created in PMML format using R and python (and Knime) but it needs three separate and unconnected steps:- 1- first step for data preparation in our Java/JVM application and save the sampling data (training and test) data in csv file or in DB, - *<JAVA/JVM BASED application>* *2- Create a machine learning Model in R and python (and Knime) and export it in PMML 4.2 format - <NON JAVA BASED >* 3- Import/deploy the PMML in our Java based application and use JPMML evaluator to execute it in production. *<JAVA BASED>* I am sure it's a common problem in machine learning as generally in Production JAVA is preferred over Python or R. Could you suggest what is the better approach(s) to *create* as well as *execute* a python/scikit based machine learning model in JVM based application. What are your thought to achieve the steps # 2 and #3 more seamlessly in a JVM based application, without compromising *performance and usability*:- 1- Call a java program which internally calls the python scikit script <http://stackoverflow.com/questions/12738827/how-can-i-call-scikit-learn-cla…>(under the hood) to create a model in PMML <https://github.com/jpmml/jpmml-sklearn> and then use JPMML evaluator. It will pretend to the user that he is in a single JVM based application (better usability). I am not sure what are the limitations and short coming of using PMML as not all features are supported in jpmml-sklearn <https://github.com/jpmml/jpmml-sklearn>. 2- Call a java program which internally calls the python script and do the model creation as well as execution in an external python environment and serialized the model and the results in a file/csv or in memory DB (or cache, like hazelcast) from where the parent Java application will fetch the results etc.. I researched that I can’t use Jython for executing Sci-kit models <http://stackoverflow.com/questions/12738827/how-can-i-call-scikit-learn-cla…> . 3- Can I use Jep <https://github.com/mrj0/jep> (Embed Python in Java) to embed Cpython in JVM ? Does anybody tried it for sci-kit models? Alternatively, I should explore to use Mahout or weka - java based machine learning libraries in my JVM based application. (I need to support both windows and non-windows platforms) I am also exploring H2Oai which is java based. Does anybody tried it. Regards Gaurav

2 1

EuroSciPy 2016 Call for Papers Extended
by federico vaggi June 8, 2016

June 8, 2016

Hi everyone, The call for contributions (talks, posters, sprints) is still open until June 24th. EuroSciPy 2016 takes place in Erlangen, Germany, from the 23 to the 27 of August and consists of two days of tutorials (beginner and advanced tracks) and two days of conference representing many fields of science, with a focus on Python tools for science. A day of sprints follows (sprints TBA). The keynote speakers are Ga?l Varoquaux (you might have heard of him) and Abby Cabunoc Mayes and we can expect a rich tutorial and scientific program! Videos from previous years are available at https://www.youtube.com/playlist?list=PLYx7XA2nY5GeQCCugyvtnHMVLdhYlrRxH and https://www.youtube.com/playlist?list=PLYx7XA2nY5Gcpabmu61kKcToLz0FapmHu We are particularly eager to receive proposals from newcomers. EuroSciPy is a very welcoming conference, and we are very curious to hear how you use Python/machine learning in your every day research. Visit us, register and submit an abstract on our website! https://www.euroscipy.org/2016/ SciPythonic regards, The EuroSciPy 2016 team

1 0

NB-SVM Implementation
by Ivo Petkantchin June 8, 2016

June 8, 2016

Hello, For a university project I worked on a Sentiment Analysis challenge (Movie Reviews) and implemented a version of NBSVM as described in this paper: http://nlp.stanford.edu/pubs/sidaw12_simple_sentiment.pdf If I am not wrong there is no NBSVM class in scikit-learn. That is why I would like to contribute by coding a NB matrix class if the work is not done by someone else already. Best regards, Ivaylo Petkantchin

2 2

Fitting Lognormal Distribution
by Startup Hire June 7, 2016

June 7, 2016

Hi all, Hope you are doing good. I am working on a project where I need to do the following things: 1. I need to fit a lognormal distribution to a set of values [I know its lognormal by a simple XY scatter plot in excel] 2. I need to find the intersection of the lognormal distribution so that I can decide cut-off values based on that. Can you guide me on (1) and (2) can be achieved in python? Regards, Sanant

8 21

memory efficient feature extraction
by Roman Yurchak June 6, 2016

June 6, 2016

Dear all, I was wondering if somebody could advise on the best way for generating/storing large sparse feature sets that do not fit in memory? In particular, I have the following workflow, Large text dataset -> HashingVectorizer -> Feature set in a sparse CSR array on disk -> Training a classifier -> Predictions where the the generated feature set is too large to fit in RAM, however the classifier training can be done in one step (as it uses only certain rows of the CSR array) and the prediction can be split in several steps, all of which fit in memory. Since the training can be performed in one step, I'm not looking for incremental learning out-of-core approaches and saving features to disk for later processing is definitely useful. For instance, if it was possible to save the output of the HashingVectorizer to a single file on disk (using e.g. joblib.dump) then load this file as a memory map (using e.g. joblib.load(.., mmap_mode='r')) everything would work great. Due to memory constraints this cannot be done directly, and the best case scenario is applying HashingVectorizer on chunks of the dataset, which produces a series of sparse CSR arrays on disk. Then, - concatenation of theses arrays into a single CSR array appears to be non-tivial given the memory constraints (e.g. scipy.sparse.vstack transforms all arrays to COO sparse representation internally). - I was not able to find an abstraction layer that would allow to represent these sparse arrays as a single array. For instance, dask could allow to do this for dense arrays ( http://dask.pydata.org/en/latest/array-stack.html ), however support for sparse arrays is only planned at this point ( https://github.com/dask/dask/issues/174 ). Finally, it is not possible to pre-allocate the full array on disk in advance (and access it as a memory map) because we don't know the number of non-zero elements in the sparse array before running the feature extraction. Of course, it is possible to overcome all these difficulties by using a machine with more memory, but my point is rather to have a memory efficient workflow. I would really appreciate any advice on this and would be happy to contribute to a project in the scikit-learn environment aiming to address similar issues, Thank you, Best, -- Roman

2 2