Mailman 3 January 2020 - scikit-learn

ANN: DIPY 1.1.1 - a powerful release
by Eleftherios Garyfallidis Jan. 22, 2020

Jan. 22, 2020

We are excited to announce a new release of DIPY: DIPY 1.1.1 is out! In addition: a) A new 5 day workshop available during March 16-20 to learn the theory and applications of the hundreds of methods available in DIPY 1.1.1 Intense! See the exquisite program here <https://workshop.dipy.org>. *b) Given the need for a myriad of new DIPY derivative projects, DIPY moved to its own organization in GitHub. **Long live DIPY! * *And therefore, *https://github.com/dipy/dipy* supersedes https://github.com/nipy/dipy <https://github.com/nipy/dipy> The old link will be available as a redirect link for the next 6 months.* c) Please support us by *citing** DIPY* in your papers using the following DOI: 10.3389/fninf.2014.00008 <https://www.ncbi.nlm.nih.gov/pubmed/24600385>otherwise the DIPY citation police will find you. ;) DIPY 1.1.1 (Friday, 10 January 2020) This release received contributions from 11 developers (the full release notes are at: https://dipy.org/documentation/1.1.1./release_notes/release1.1/). Thank you all for your contributions and feedback! Please click here <https://dipy.org/documentation/1.1.1./api_changes/> to check API changes. Highlights of this release include: - New module for deep learning DIPY.NN (uses TensorFlow 2.0). - Improved DKI performance and increased utilities. - Non-linear and RESTORE fits from DTI compatible now with DKI. - Numerical solutions for estimating axial, radial and mean kurtosis. - Added Kurtosis Fractional Anisotropy by Glenn et al. 2015. - Added Mean Kurtosis Tensor by Hansen et al. 2013. - Nibabel minimum version is 3.0.0. - Azure CI added and Appveyor CI removed. - New command line interfaces for LPCA, MPPCA and Gibbs Unringing. - New MTMS CSD tutorial added. - Horizon refactored and updated to support StatefulTractograms. - Speeded up all cython modules by using a smarter configuration setting. - All tutorials updated to API changes and 2 new tutorials added. - Large documentation update. - Closed 126 issues and merged 50 pull requests. Note: - Have in mind that DIPY stopped supporting Python 2 after version 0.16.0. All major Python projects have switched to Python 3. It is time that you switch too. To upgrade or install <http://dipy.org/release0.10.html> DIPY Run the following command in your terminal: <http://dipy.org/release0.10.html> pip install --upgrade dipy or conda install -c conda-forge dipy This version of DIPY depends on nibabel (3.0.0+). For visualization you need FURY (0.4.0+). Questions or suggestions? For any questions go to http://dipy.org, or send an e-mail to dipy(a)python.org <neuroimaging(a)python.org> We also have an instant messaging service and chat room available at https://gitter.im/nipy/dipy On behalf of the DIPY developers, Eleftherios Garyfallidis, Ariel Rokem, Serge Koudoro https://dipy.org/contributors

3 3

ask a question about weights for features in svc with rbf kernel
by Rujing Zha Jan. 22, 2020

Jan. 22, 2020

Hi experts and users, I am going to extact the pattern of svc. But I do not know how to extract weights for each feature using this svc classifiers with kernel of rbf function. Thank you. Rujing

3 3

Why is subset invariance necessary for transfom()?
by Charles Pehlivanian Jan. 21, 2020

Jan. 21, 2020

Not all data transformers have a transform method. For those that do, subset invariance is assumed as expressed in check_methods_subset_invariance(). It must be the case that T.transform(X)[i] == T.transform(X[i:i+1]), e.g. This is true for classic projections - PCA, kernel PCA, etc., but not for some manifold learning transformers - MDS, SpectralEmbedding, etc. For those, an optimal placement of the data in space is a constrained optimization, may take into account the centroid of the dataset etc. The manifold learners have "batch" oos transform() methods that aren't implemented, and wouldn't pass that test. Instead, those that do - LocallyLinearEmbedding - use a pointwise version, essentially replacing a batch fit with a suboptimal greedy one [for LocallyLinearEmbedding]: for i in range(X.shape[0]): X_new[i] = np.dot(self.embedding_[ind[i]].T, weights[i]) Where to implement the batch transform() methods for MDS, SpectralEmbedding, LocallyLinearEmbedding, etc? Another verb? Both batch and pointwise versions? The latter is easy to implement once the batch version exists. Relax the test conditions? transform() is necessary for oos testing, so necessary for cross validation. The batch versions should be preferred, although as it stands, the pointwise versions are. Thanks Charles Pehlivanian

2 1

Heisenbug?
by Dan Stromberg Jan. 17, 2020

Jan. 17, 2020

Hi folks. I'm new to Scikit-learn. I have a very large Python project that seems to have a heisenbug which is manifesting in scikit-learn code. Short of constructing an SSCCE, are there any magical techniques I should try for pinning down the precise cause? Like valgrind or something? An SSCCE will most likely be pretty painful: the project has copious shared, mutable state, and I've already tried a largish test program that calls into the same code path with the error manifesting 0 times in 100. It's quite possible the root cause will turn out to be some other part of the software stack. The traceback from pytest looks like: sequential/test_training.py:101: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../rt/classifier/coach.py:146: in train **self.classifier_section ../domain/classifier/factories/classifier_academy.py:115: in create_classifier **kwargs) ../domain/classifier/factories/imp/xgb_factory.py:164: in create clf_random.fit(X_train, y_train) ../../../../.local/lib/python3.6/site-packages/sklearn/model_selection/_search.py:722: in fit self._run_search(evaluate_candidates) ../../../../.local/lib/python3.6/site-packages/sklearn/model_selection/_search.py:1515: in _run_search random_state=self.random_state)) ../../../../.local/lib/python3.6/site-packages/sklearn/model_selection/_search.py:711: in evaluate_candidates cv.split(X, y, groups))) ../../../../.local/lib/python3.6/site-packages/sklearn/externals/joblib/parallel.py:996: in __call__ self.retrieve() ../../../../.local/lib/python3.6/site-packages/sklearn/externals/joblib/parallel.py:899: in retrieve self._output.extend(job.get(timeout=self.timeout)) ../../../../.local/lib/python3.6/site-packages/sklearn/externals/joblib/_parallel_backends.py:517: in wrap_future_result return future.result(timeout=timeout) /usr/lib/python3.6/concurrent/futures/_base.py:425: in result return self.__get_result() _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <Future at 0x7f15571ec7f0 state=finished raised ValueError> def __get_result(self): if self._exception: > raise self._exception E ValueError: Input contains NaN, infinity or a value too large for dtype('float32'). /usr/lib/python3.6/concurrent/futures/_base.py:384: ValueError The above exception is raised about 12 to 14 times in 100 in full-blown automated testing. Thanks for the cool software.

2 5

Issues for Berlin and Paris Sprints
by Chiara Marmo Jan. 17, 2020

Jan. 17, 2020

Dear core-devs, First let me wish a Happy New Year to you all! There will be two scikit-learn sprints in January to start this 2020 in a busy way: one in Berlin [1] (Jan 25) and one in Paris [2] (Jan 28-31). I feel like we could benefit of some coordination in selecting the issues for those two events. Reshama Shaikh and I, we are already in touch. I've opened two projects [3][4] to follow-up the issue selection for the sprints. I will check for previous "Sprint" labels in the skl issues and maybe ask for clarification on some of them... please, be patient. The goal is to prepare the two sprints in order to make the review process as efficient as possible: we don't want to waste the reviewer time and we hope to make the PR experience a learning opportunity on both sides. In particular, I would like to ask a favour to all of you: I don't know if this is even always possible, but, IMO, it would be really useful to have a list of two/three reviewers available to check on a specific issue. I am, personally, a bit uncomfortable in pinging core-devs randomly, under the impression of crying wolf lacking for attention... If people in charge are defined in advance this could, I think, smooth the review process. What do you think? Please, let us know if you have any suggestion or recommendation to improve the Sprint organization. Thanks for listening, Best, Chiara [1] https://github.com/WiMLDS/berlin-2020-scikit-sprint [2] https://github.com/scikit-learn/scikit-learn/wiki/Paris-scikit-learn-Sprint… [3] https://github.com/WiMLDS/berlin-2020-scikit-sprint/projects/1 [4] https://github.com/scikit-learn-fondation/ParisSprintJanuary2020/projects/1

2 2

Attribute Incremental learning
by Farzana Anowar Jan. 16, 2020

Jan. 16, 2020

Hello, This is Farzana. I am trying to understand the attribute incremental learning ( or virtual concept drift) which is every time when a new feature will be available for a real-time dataset (i.e. any online auction dataset) a classifier will add that new feature with the existing features in a dataset and classify the new dataset (with previous features and new features) incrementally. I know that we can convert a static classifier to an incremental classifier in scikit-learn. However, I could not find any library or function for attribute incremental learning or any detail information. It would be great if anyone could give me some insight on this. Thanks! -- Best Regards, Farzana Anowar, PhD Candidate Department of Computer Science University of Regina

2 2

Decision tree call chronology
by aditya aggarwal Jan. 14, 2020

Jan. 14, 2020

Hello I am trying to understand the order of functions call for performing classification using decision tree in sklearn. I need to make and test some changes in the algorithm used to calculate best split for my dissertation. I have looked up the documentation available of sklearn and other sources available online but couldn't seem to crack it. Any help would be appreciated. Thanks

2 1

Changes in the classifier
by aditya aggarwal Jan. 13, 2020

Jan. 13, 2020

Hello I'm trying to change the entropy function which is used in sklearn for DecisionTreeClassification locally on my system. when I rerun the pip install --editable . command after updating the cython file, I receive the following error message: Error compiling Cython file: ------------------------------------------------------------ ... for k in range(self.n_outputs): for c in range(n_classes[k]): count_k = sum_total[c] if count_k > 0.0: count_k /= self.weighted_n_node_samples entropy -= count_k * np.log2(count_k) ^ ------------------------------------------------------------ sklearn/tree/_criterion.pyx:537:20: Coercion from Python not allowed without the GIL This error is persisten with other errors as: Operation not allowed without gil Converting to Python object not allowed without gil Converting to Python object not allowed without gil Calling gil-requiring function not allowed without gil Accessing Python attribute not allowed without gil Accessing Python global or builtin not allowed without gil I've tried looking up for solution on various sites, but could not resolve the issue. Any help would be appreciated. Thanks and regards Aditya Aggarwal

2 2

logistic regression results are not stable between solvers
by Benoît Presles Jan. 9, 2020

Jan. 9, 2020

Dear scikit-learn users, I am using logistic regression to make some predictions. On my own data, I do not get the same results between solvers. I managed to reproduce this issue on synthetic data (see the code below). All solvers seem to converge (n_iter_ < max_iter), so why do I get different results? If results between solvers are not stable, which one to choose? Best regards, Ben ------------------------------------------ Here is the code I used to generate synthetic data: from sklearn.datasets import make_classification from sklearn.model_selection import StratifiedShuffleSplit from sklearn.preprocessing import StandardScaler from sklearn.linear_model import LogisticRegression # RANDOM_SEED = 2 # X_sim, y_sim = make_classification(n_samples=200, n_features=45, n_informative=10, n_redundant=0, n_repeated=0, n_classes=2, n_clusters_per_class=1, random_state=RANDOM_SEED, shuffle=False) # sss = StratifiedShuffleSplit(n_splits=10, test_size=0.2, random_state=RANDOM_SEED) for train_index_split, test_index_split in sss.split(X_sim, y_sim): X_split_train, X_split_test = X_sim[train_index_split], X_sim[test_index_split] y_split_train, y_split_test = y_sim[train_index_split], y_sim[test_index_split] ss = StandardScaler() X_split_train = ss.fit_transform(X_split_train) X_split_test = ss.transform(X_split_test) # classifier_lbfgs = LogisticRegression(fit_intercept=True, max_iter=20000000, verbose=1, random_state=RANDOM_SEED, C=1e9, solver='lbfgs') classifier_lbfgs.fit(X_split_train, y_split_train) print('classifier lbfgs iter:', classifier_lbfgs.n_iter_) classifier_saga = LogisticRegression(fit_intercept=True, max_iter=20000000, verbose=1, random_state=RANDOM_SEED, C=1e9, solver='saga') classifier_saga.fit(X_split_train, y_split_train) print('classifier saga iter:', classifier_saga.n_iter_) # y_pred_lbfgs = classifier_lbfgs.predict(X_split_test) y_pred_saga = classifier_saga.predict(X_split_test) # if (y_pred_lbfgs==y_pred_saga).all() == False: print('lbfgs does not give the same results as saga :-( !') exit()

5 17

Why ridge regression can solve multicollinearity?
by lampahome Jan. 9, 2020

Jan. 9, 2020

I find out many blogs said that the l2 regularization solve multicollinearity, but they don't said how it works. I thought LASSO is able to select features by l1 regularization, maybe it also can solve this. Can anyone tell me how ridge works with multicollinearity great? thx

4 5