Mailman 3 May 2019 - scikit-learn

ANN: scikit-learn 0.21.2 released
by Olivier Grisel May 25, 2019

May 25, 2019

A quick bugfix release to fix a critical regression in the computation of the euclidean distances returning incorrect values silently. This release also includes other bugfixes listed in the changelog: https://scikit-learn.org/0.21/whats_new.html#version-0-21-2 The PyPI.org wheels and conda-forge packages are online. The packages for the default Anaconda channel should follow soon. Thanks to all the contributors! -- Olivier http://twitter.com/ogrisel - http://github.com/ogrisel

2 1

ANN: Scikit-learn 0.21.2 released
by Joel Nothman May 25, 2019

May 25, 2019

We've released 0.21.2 primarily to fix an issue with euclidean_distances (and pairwise_distances). It should be available on PyPI and Conda-Forge. Full list of changes at https://scikit-learn.org/0.21/whats_new/v0.21.html Thanks to all who helped fix these issues so quickly after 0.21.1.

1 0

Highly cited paper - causal random forests
by Randy Ellis May 25, 2019

May 25, 2019

Would this be difficult for a moderate user to implement in sklearn by modifying the existing code base? Estimation and Inference of Heterogeneous Treatment Effects using Random Forests 342 citations in less than a year (Google Scholar): https://amstat.tandfonline.com/doi/full/10.1080/01621459.2017.1319839 "In this article, we develop a nonparametric *causal forest* for estimating heterogeneous treatment effects that extends Breiman’s widely used random forest algorithm. In the potential outcomes framework with unconfoundedness, we show that causal forests are pointwise consistent for the true treatment effect and have an asymptotically Gaussian and centered sampling distribution. We also discuss a practical method for constructing asymptotic confidence intervals for the true treatment effect that are centered at the causal forest estimates. Our theoretical results rely on a generic Gaussian theory for a large family of random forest algorithms. To our knowledge, this is the first set of results that allows any type of random forest, including classification and regression forests, to be used for provably valid statistical inference. In experiments, we find causal forests to be substantially more powerful than classical methods based on nearest-neighbor matching, especially in the presence of irrelevant covariates." -- *Randall J. Ellis* PhD Student, Hurd lab <http://labs.neuroscience.mssm.edu/project/hurd-lab/>, Mount Sinai School of Medicine Special Volunteer, Michaelides lab <http://www.michaelideslab.org/>, NIDA IRP Phone: +1-954-260-9891

2 1

[Copyright] Skicit-learn graphic
by Rätz, Martin May 24, 2019

May 24, 2019

Dear scikit-learn employees, on the scikit-learn webpage (link to the graphic<https://scikit-learn.org/stable/auto_examples/model_selection/plot_underfit…>) you will find a graphic, which I would like to use in a publication in an international journal. I slightly modified the graphic as you can see in the appendix. Of course, I refer to Scikit-learn in the caption. The naming in the bibliography is as follows: F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and É. Duchesnay, "Scikit-learn: Machine learning in python," Journal of Machine Learning Research, vol. 12, no. Oct, pp. 2825- 2830, 2011. I would like to kindly ask for permission to publish the graphic. Yours sincerely Martin Rätz _______________________________________ Martin Rätz, M.Sc. Research Associate T +49 241 80-49794 F +49 241 80-49769 mraetz(a)eonerc.rwth-aachen.de<mailto:mraetz@eonerc.rwth-aachen.de> RWTH Aachen University E.ON Energy Research Center * Institute for Energy Efficient Buildings and Indoor Climate E.ON Energieforschungszentrum * Lehrstuhl für Gebäude- und Raumklimatechnik Mathieustraße 10 52074 Aachen * Germany www.eonerc.rwth-aachen.de/ebc<http://www.eonerc.rwth-aachen.de/ebc>

2 1

decision_path method for tree-based models
by Beaugnon Anael May 23, 2019

May 23, 2019

Hi everyone, The decision_path method is currently available only for DecisionTreeClassifier, DecisionTreeRegressor, and RandomForest, but not for IsolationForest and GradientBoostingClassifier. In these cases, the implementation is quite easy, it is exactly the same as for RandomForest, but it I think it would be very handy to have a public method. What do you think of this proposal ? If you are ok with it, I would be happy to propose a pull request. Thanks, -- Anaël Beaugnon ANSSI - Intrusion Detection Research Laboratory Les données à caractère personnel recueillies et traitées dans le cadre de cet échange, le sont à seule fin d’exécution d’une relation professionnelle et s’opèrent dans cette seule finalité et pour la durée nécessaire à cette relation. Si vous souhaitez faire usage de vos droits de consultation, de rectification et de suppression de vos données, veuillez contacter contact.rgpd(a)sgdsn.gouv.fr. Si vous avez reçu ce message par erreur, nous vous remercions d’en informer l’expéditeur et de détruire le message. The personal data collected and processed during this exchange aims solely at completing a business relationship and is limited to the necessary duration of that relationship. If you wish to use your rights of consultation, rectification and deletion of your data, please contact: contact.rgpd(a)sgdsn.gouv.fr. If you have received this message in error, we thank you for informing the sender and destroying the message.

3 2

Regularization in Tree Models
by prudhviraj nitjsr May 22, 2019

May 22, 2019

Hi All, I've noticed that there is no regularization term for Decision Tree estimators in scikit learn. Are there any plans to introduce that? Thanks Prudvi RajKumar M

2 1

Introducing creme for online learning
by Max Halford May 16, 2019

May 16, 2019

Hello everyone, I sometimes see emails where people are asking about training models incrementally. Me and some friends have started a Python library for doing so-called online learning named creme: https://github.com/creme-ml/creme. The code is idiomatic and the API resembles that of sklearn. Online learning is treated as a first class citizen, which makes more practical and efficient than sklearn if online learning is your goal. Each estimator has a fit_one(x, y) method which allows it to train with one observation at a time. I just presented it at PyData Amsterdam where people seemed enthusiastic about it. The video is not out yet but here are the slides: https://maxhalford.github.io/slides/creme-pydata. Best regards. And congrats on version 0.21! -- Max Halford +336 28 25 13 38

1 0

ANN: scikit-learn 0.21 released
by Joel Nothman May 16, 2019

May 16, 2019

Thanks to the work of many, many contributors, we have released Scikit-learn 0.21. It is available from GitHub, PyPI and Conda-forge, but is not yet available on the Anaconda defaults channel. * Documentation at https://scikit-learn.org/0.21 * Release Notes at https://scikit-learn.org/0.21/whats_new * Download source or wheels at https://pypi.org/project/scikit-learn/0.21rc2/ * Install from conda-forge with `conda install -c conda-forge scikit-learn` Highlights include: * neighbors.NeighborhoodComponentsAnalysis for supervised metric learning, which learns a weighted euclidean distance for k-nearest neighbors. https://scikit-learn.org/0.21/modules/neighbors.html#nca * ensemble.HistGradientBoostingClassifier and ensemble.HistGradientBoostingRegressor: experimental implementations of efficient binned gradient boosting machines. https://scikit-learn.org/0.21 /modules/ensemble.html#gradient-tree-boosting * impute.IterativeImputer: an experimental API for a non-trivial approach to missing value imputation. https://scikit-learn.org/0.21 /modules/impute.html#multivariate-feature-imputation * cluster.OPTICS: a new density-based clustering algorithm. https://scikit-learn.org/0.21/modules/clustering.html#optics * better printing of estimators as strings, with an option to hide default parameters for compactness: https://scikit-learn.org/0.21 /auto_examples/plot_changed_only_pprint_parameter.html * for estimator and library developers: a way to tag your estimator so that it can be treated appropriately with check_estimator. https://scikit-learn.org/0.21/developers/contributing.html#estimator-tags There are many other enhancements and fixes listed in the release notes ( https://scikit-learn.org/0.21/whats_new). Please note that Scikit-learn has new dependencies. It requires: * joblib >= 0.11, which used to be vendored within Scikit-learn * OpenMP, unless the environment variable SKLEARN_NO_OPENMP=1 when the code is compiled (and cythonized) * Python >= 3.5. Installing Scikit-learn from Python 2 will continue to provide version 0.20. Thanks again to everyone who contributed and to our sponsors, who helped us to develop such a great set of features and fixes since version 0.20 in under 8 months. Happy Learning! >From the Scikit-learn ]team.

3 2

Can I evaluate clustering efficiency incrementally?
by lampahome May 16, 2019

May 16, 2019

I see some algo can cluster incrementally if dataset is too huge ex: minibatchkmeans and Birch. But is there any way to evaluate incrementally? I found silhouette-coefficient and Calinski-Harabaz index because I don't know the ground truth labels. But they can't evaluate incrementally.

5 9

Example of a scikit-learn compatible classifier with C++ implementation of the algorithms
by drh＠aiwerkstatt.com May 15, 2019

May 15, 2019

I use a PYTHON BASED ECOSYSTEM (SCIKIT-LEARN, … ) FOR PROTOTYPING and I have a C++ BASED PRODUCTION SYSTEM. A scikit-learn compatible interface allows me to take advantage of scikit-learn’s ecosystem. Implementing the algorithm in C++ allows me to develop and test my algorithms already during prototyping. I started with scikit-learn’s project template to roll my own decision tree and forest classifier and implemented the algorithms in a C++ library, using Cython to create the Python bindings. Starting out with a Python implementation, I experimented a little bit with implementing the algorithms in Cython. But I found that if you are proficient in Python and C++ coding, that implementing the algorithm directly in C++ was much faster than writing it in Cython. I made this project available to everybody, because I think it could serve as an example or template for anybody who would like to roll their own scikit-learn compatible classifier with a C++ based implementation of the algorithms to be re-used in a production system. At least version 1.0.0 should be useful, after that it might become too complex to be used as an example. Check it out: READTHEDOCs: https://koho.readthedocs.io GITHUB: https://github.com/AIWerkstatt/koho I tried to be consistent with scikit-learn’s decision tree and ensemble modules, and the basic concepts, including stack, samples LUT with in-place partitioning, incremental histogram updates, for the implementation of the classifiers are based on: G. Louppe, Understanding Random Forests, PhD Thesis, 2014. Thanks a lot Gilles for that comprehensive work on random forests!

1 0