Mailman 3 September 2016 - scikit-learn

Joining the Community
by Josh Karnofsky SEAS Sept. 15, 2016

Sept. 15, 2016

Hi everyone, My name’s Josh and I am a student at the University of Pennsylvania. I’m joining the scikit-learn community this semester as part of a computer science course on open source software. I’m excited to work with everyone. - Josh K

2 1

Scikit-learn 0.18-rc2 release candidate!
by Andreas Mueller Sept. 15, 2016

Sept. 15, 2016

Hi all. We just published the 0.18-rc2 release candidate on pipy and anaconda.org. Please go ahead and test it, so we can iron out the issues for the 0.18 release. We plan to release in the next ~2 weeks. You can find all the amazing new features here: http://scikit-learn.org/dev/whats_new.html The stable website will be updated after the final release. You can upgrade to the release candidate as follows. With pip: pip install scikit-learn==0.18.rc2 If you're using conda-forge, you can also update using conda |*conda* install -c https://conda.anaconda.org/conda-forge/label/rc scikit-learn=0.18rc2| Don't use pip to install a package that you installed with conda (or via anaconda!). If you're using conda but not conda-forge, please completely remove your scikit-learn installation and then install with pip (or wait for the full release). A big thank you to everybody who contributed. All the best, Andy

10 16

Re: [scikit-learn] Use of Scaler with LassoCV, RidgeCV
by Brenet, Yoann Sept. 14, 2016

Sept. 14, 2016

Hi Andreas, Thanks a lot for the information. Yoann Date: Tue, 13 Sep 2016 11:56:45 -0400 From: Andreas Mueller <t3kcit(a)gmail.com> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] Use of Scaler with LassoCV, RidgeCV Message-ID: <17283c24-44b3-4f52-3b7b-7824f1a4c60b(a)gmail.com> Content-Type: text/plain; charset=windows-1252; format=flowed There is no way to use the "efficient" EstimatorCV objects with pipelines. This is an API bug and there's an open issue and maybe even a PR for that. On 09/13/2016 08:45 AM, Brenet, Yoann wrote: > Hi Sebastian, > > Many thanks, that's what I was thinking I should be doing, so thanks a lot for confirming that was the way to go. > > Really appreciate the help, > Yoann > > Date: Tue, 13 Sep 2016 08:33:52 -0400 > From: Sebastian Raschka <se.raschka(a)gmail.com> > To: Scikit-learn user and developer mailing list > <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] Use of Scaler with LassoCV, RidgeCV > Message-ID: <65093BCA-70D7-4DCF-9EDC-C4EA7B513C3C(a)gmail.com> > Content-Type: text/plain; charset=utf-8 > > Hi, Yoann, > > when I understand correctly, you want to apply the scaling to each iteration in cross-validation (i.e., the recommended way to do it)? Here, you could use the make_pipeline function, which will call fit on each training fold and call transform on each test fold: > > > from sklearn.preprocessing import StandardScaler from sklearn.pipeline > import make_pipeline from sklearn.cross_validation import > cross_val_score from sklearn.linear_model import Ridge > > pipe = make_pipeline(StandardScaler(), Ridge()) cross_val_score(pipe, > X, y) > > You can think of ?pipe? as an Ridge estimator with a StandardScaler attached to it. > > Best, > Sebastian > > >> On Sep 13, 2016, at 8:16 AM, Brenet, Yoann <yoann.brenet(a)se1.bp.com> wrote: >> >> Hi all, >> >> I was trying to use scikit-learn LassoCV/RidgeCV while applying a 'StandardScaler' on each fold set. I do not want to apply the scaler before the cross-validation to avoid leakage but I cannot figure out how I am supposed to do that with LassoCV/RidgeCV. >> >> Is there a way to do this ? Or should I create a pipeline with Lasso/Ridge and 'manually' search for the hyper-parameters (using GridSearchCV for instance) ? >> >> Many thanks. >> >> Yoann >> _______________________________________________ >> scikit-learn mailing list >> scikit-learn(a)python.org >> https://mail.python.org/mailman/listinfo/scikit-learn > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn > > > ------------------------------ > > End of scikit-learn Digest, Vol 6, Issue 15 > ******************************************* > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn ------------------------------ Subject: Digest Footer _______________________________________________ scikit-learn mailing list scikit-learn(a)python.org https://mail.python.org/mailman/listinfo/scikit-learn ------------------------------ End of scikit-learn Digest, Vol 6, Issue 17 *******************************************

1 0

Re: [scikit-learn] Use of Scaler with LassoCV, RidgeCV
by Brenet, Yoann Sept. 13, 2016

Sept. 13, 2016

Hi Sebastian, Many thanks, that's what I was thinking I should be doing, so thanks a lot for confirming that was the way to go. Really appreciate the help, Yoann Date: Tue, 13 Sep 2016 08:33:52 -0400 From: Sebastian Raschka <se.raschka(a)gmail.com> To: Scikit-learn user and developer mailing list <scikit-learn(a)python.org> Subject: Re: [scikit-learn] Use of Scaler with LassoCV, RidgeCV Message-ID: <65093BCA-70D7-4DCF-9EDC-C4EA7B513C3C(a)gmail.com> Content-Type: text/plain; charset=utf-8 Hi, Yoann, when I understand correctly, you want to apply the scaling to each iteration in cross-validation (i.e., the recommended way to do it)? Here, you could use the make_pipeline function, which will call fit on each training fold and call transform on each test fold: from sklearn.preprocessing import StandardScaler from sklearn.pipeline import make_pipeline from sklearn.cross_validation import cross_val_score from sklearn.linear_model import Ridge pipe = make_pipeline(StandardScaler(), Ridge()) cross_val_score(pipe, X, y) You can think of ?pipe? as an Ridge estimator with a StandardScaler attached to it. Best, Sebastian > On Sep 13, 2016, at 8:16 AM, Brenet, Yoann <yoann.brenet(a)se1.bp.com> wrote: > > Hi all, > > I was trying to use scikit-learn LassoCV/RidgeCV while applying a 'StandardScaler' on each fold set. I do not want to apply the scaler before the cross-validation to avoid leakage but I cannot figure out how I am supposed to do that with LassoCV/RidgeCV. > > Is there a way to do this ? Or should I create a pipeline with Lasso/Ridge and 'manually' search for the hyper-parameters (using GridSearchCV for instance) ? > > Many thanks. > > Yoann > _______________________________________________ > scikit-learn mailing list > scikit-learn(a)python.org > https://mail.python.org/mailman/listinfo/scikit-learn ------------------------------ Subject: Digest Footer _______________________________________________ scikit-learn mailing list scikit-learn(a)python.org https://mail.python.org/mailman/listinfo/scikit-learn ------------------------------ End of scikit-learn Digest, Vol 6, Issue 15 *******************************************

2 2

Problems with plotting decision regions
by Sebastian Raschka Sept. 13, 2016

Sept. 13, 2016

Hi, all, I am having some problems with showing decision regions in 2D if more than 4 classes are present. Really can’t figure out the source of the problem and would really appreciate some help if you have done this before or have any pointer since I am afraid that I am overlooking something really simple here. I created a gist of 2 simple examples with images attached: https://gist.github.com/rasbt/6fb65bba38b70e28e60a9842b988cc67 I think it is very likely that it is not a bug in scikit-learn but rather a matplotlib contourf bug? In case it is a bug at all … Best, Sebastian

2 2

is RandomForest random samples or random features?
by 斌洪 Sept. 13, 2016

Sept. 13, 2016

I have read the Guide of sklearn's RandomForest : """ In random forests (see RandomForestClassifier <http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomFor…> and RandomForestRegressor <http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomFor…> classes), each tree in the ensemble is built from a sample drawn with replacement (i.e., a bootstrap sample) from the training set. """ But I prefer RandomForest as : """ features ("attributes", "predictors", "independent variables") are randomly sampled """ is RandomForest random samples or random features? where can I find a features random version of RandomForest? thx.

3 4

Use of Scaler with LassoCV, RidgeCV
by Brenet, Yoann Sept. 13, 2016

Sept. 13, 2016

Hi all, I was trying to use scikit-learn LassoCV/RidgeCV while applying a 'StandardScaler' on each fold set. I do not want to apply the scaler before the cross-validation to avoid leakage but I cannot figure out how I am supposed to do that with LassoCV/RidgeCV. Is there a way to do this ? Or should I create a pipeline with Lasso/Ridge and 'manually' search for the hyper-parameters (using GridSearchCV for instance) ? Many thanks. Yoann

3 2

Gradient Boosting: Feature Importances do not sum to 1
by Douglas Chan Sept. 13, 2016

Sept. 13, 2016

Hello everyone, I notice conditions when Feature Importance values do not add up to 1 in ensemble tree methods, like Gradient Boosting Trees or AdaBoost Trees. I wonder if there’s a bug in the code. This error occurs when the ensemble has a large number of estimators. The exact conditions depend variously. For example, the error shows up sooner with a smaller amount of training samples. Or, if the depth of the tree is large. When this error appears, the predicted value seems to have converged. But it’s unclear if the error is causing the predicted value not to change with more estimators. In fact, the feature importance sum goes lower and lower with more estimators thereafter. I wonder if we’re hitting some floating point calculation error. Looking forward to hear your thoughts on this. Thank you! -Doug

3 5

pomegranate v0.6.0 release
by Jacob Schreiber Sept. 10, 2016

Sept. 10, 2016

Hello everyone! I just released pomegranate v0.6.0, which focuses on probabilistic modelling for python. It currently implements basic distributions, naive bayes, markov chains, general mixture models, hidden Markov models, and Bayesian networks in a fast and extremely flexible manner. I have a more in depth reddit post on it. <https://www.reddit.com/r/Python/comments/52424a/pomegranate_v060_released_p…> The gist is that since the last version, I've added model stacking (mixtures of HMMs, Naive Bayes of Bayesian Networks or mixture models....), native parallelization through joblib, extended the out-of-core API to include all models and model stacks, and significantly increased the speed using a BLAS backend. I recently gave a talk at PyData Chicago about pomegranate and released an in depth notebook tutorial <https://github.com/jmschrei/pomegranate/blob/master/tutorials/PyData_2016_C…> covering all the cool new features which you should check out. In addition, I just wrote a new tutorial on how to utilize parallelization <https://github.com/jmschrei/pomegranate/blob/master/tutorials/Tutorial_7_Pa…> to train a Gaussian mixture model, a GMM-HMM, and a mixture of GMM-HMMs (a GMM-HMM-GMM if you will) without having to think too much about the underlying algorithms. I'd love for you all to check it out and let me know if you have any feedback or want to chat about it. Thanks! Jacob

1 0

Issue with sklearn.neural_network
by Ibrahim Dalal Sept. 9, 2016

Sept. 9, 2016

Dear Developers, I am using sklearn version 0.17.1 on Ubuntu 14.04. I was checking out neural network examples and one such example used sklearn.neural_network.MLPClassifier. When I tried this, I get the following error: >>> from sklearn import neural_network >>> clf = neural_network.MLPClassifier() Traceback (most recent call last): File "<stdin>", line 1, in <module> AttributeError: 'module' object has no attribute 'MLPClassifier' Thanks

4 4