Mailman 3 October 2019 - scikit-learn

Sparse Input for HistGradientBoostingClassifier
by Jason Wolosonovich Oct. 22, 2019

Oct. 22, 2019

Hi! I'm getting an error when trying to use the HistGradientBoostingClassifier by feeding it the output from CountVectorizer and then TfidfTransformer. The error is: TypeError: A sparse matrix was passed, but dense data is required. Use X.toarray() to convert to a dense numpy array. I haven't opened an issue yet because I wanted to get more clarification on whether this just isn't implemented yet or if there is some reason inherent to histogram based boosting that prevents sparse inputs from being used. Making the array dense in my case causes me to run out of memory. Thanks in advance! -Jason

2 1

question
by ahmad qassemi Oct. 20, 2019

Oct. 20, 2019

Dear Mr/Mrs, I'm a PhD student in DS. I'm trying to use your provided code on *Spectral CoClustering *and *Spectral Biclustering* to bi-cluster my data matrix ( https://scikit-learn.org/stable/modules/biclustering.html). Since my data has complex values, i.e., matrix elements are complex, your modules don't work on my data. It seems that the reason is your K-means' code doesn't work with complex numbers. I will really appreciate it if you take some time and tell me how should I apply your codes on my complex data. Thanks a lot in advance. Sincerely, Ahmad Qassemi

4 4

Weighted Random Forest vs. "class_weight" in RandomForestClassifier
by Kristen M. Altenburger Oct. 16, 2019

Oct. 16, 2019

Hi All, Posted the same question on StackExchange [link<https://stats.stackexchange.com/questions/431777/class-weight-in-random-for…>] but also circulating here to see if someone knows :) I am confused whether the "class_weight" parameter in Python's sklearn's Random Forest Classifier (https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomFo…) is equivalent to Chen/Breiman's notion of "Weighted Random Forest" described in Section 2.3 (https://statistics.berkeley.edu/sites/default/files/tech-reports/666.pdf). In short, "Weighted Random Forest" will "...assign a weight to each class, with the minority class given larger weight (i.e., higher misclassification cost). The class weights are incorporated into the RF algorithm in two places. In the tree induction procedure, class weights are used to weight the Gini criterion for finding splits. In the terminal nodes of each tree, class weights are again taken into consideration. The class prediction of each terminal node is determined by “weighted majority vote”; i.e., the weighted vote of a class is the weight for that class times the number of cases for that class at the terminal node. The final class prediction for RF is then determined by aggregatting the weighted vote from each individual tree, where the weights are average weights in the terminal nodes." Question: I can't tell from the Python source code for RandomForestClassifier, is class_weight used to weight the Gini criterion for finding splits? And if not, can anyone recommend code that implements Weighted Random Forest? Thanks! Thanks! Kristen http://kaltenburger.github.io/

1 0

Re: [scikit-learn] scikit-learn Digest, Vol 43, Issue 25
by Mike Smith Oct. 16, 2019

Oct. 16, 2019

"Second complexity does not > imply better prediction. " Complexity doesn't imply prediction? Perhaps you're having a translation error. On Sat, Oct 12, 2019 at 2:04 PM <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: scikit-learn Digest, Vol 43, Issue 24 (Mike Smith) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 12 Oct 2019 14:04:12 -0700 > From: Mike Smith <javaeurusd(a)gmail.com> > To: scikit-learn(a)python.org > Subject: Re: [scikit-learn] scikit-learn Digest, Vol 43, Issue 24 > Message-ID: > <CAEWZffD-hNviFkyxuM8CgDR3XSWOyn= > 4LRy2NJvjwvVr4RgobQ(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > "... > If I should expect good results on a pc, scikit says that needing > gpu power is > > obsolete, since certain scikit models perform better (than ml designed > for gpu) > > that are not designed for gpu, for that reason. Is this true?" > > Where do you see this written? I think that you are looking for overly > simple stories that you are not true." > > Gael, see the below from the scikit-learn FAQ. You can also find this > yourself at the main FAQ: > > [image: 2019-10-12 14_00_05-Frequently Asked Questions ? scikit-learn > 0.21.3 documentation.png] > > > On Sat, Oct 12, 2019 at 9:03 AM <scikit-learn-request(a)python.org> wrote: > > > Send scikit-learn mailing list submissions to > > scikit-learn(a)python.org > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://mail.python.org/mailman/listinfo/scikit-learn > > or, via email, send a message with subject or body 'help' to > > scikit-learn-request(a)python.org > > > > You can reach the person managing the list at > > scikit-learn-owner(a)python.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of scikit-learn digest..." > > > > > > Today's Topics: > > > > 1. Re: Is scikit-learn implying neural nets are the best > > regressor? (Gael Varoquaux) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Fri, 11 Oct 2019 13:34:33 -0400 > > From: Gael Varoquaux <gael.varoquaux(a)normalesup.org> > > To: Scikit-learn mailing list <scikit-learn(a)python.org> > > Subject: Re: [scikit-learn] Is scikit-learn implying neural nets are > > the best regressor? > > Message-ID: <20191011173433.bbywiqnwjjpvsi4r(a)phare.normalesup.org> > > Content-Type: text/plain; charset=iso-8859-1 > > > > On Fri, Oct 11, 2019 at 10:10:32AM -0700, Mike Smith wrote: > > > In other words, according to that arrangement, is scikit-learn implying > > that > > > section 1.17 is the best regressor out of the listed, 1.1 to 1.17? > > > > No. > > > > First they are not ordered in order of complexity (Naive Bayes is > > arguably simpler than Gaussian Processes). Second complexity does not > > imply better prediction. > > > > > If I should expect good results on a pc, scikit says that needing gpu > > power is > > > obsolete, since certain scikit models perform better (than ml designed > > for gpu) > > > that are not designed for gpu, for that reason. Is this true? > > > > Where do you see this written? I think that you are looking for overly > > simple stories that you are not true. > > > > > How much hardware is a practical expectation for running the best > > > scikit models and getting the best results? > > > > This is too vague a question for which there is no answer. > > > > Ga?l > > > > > On Fri, Oct 11, 2019 at 9:02 AM <scikit-learn-request(a)python.org> > wrote: > > > > > Send scikit-learn mailing list submissions to > > > ? ? ? ? scikit-learn(a)python.org > > > > > To subscribe or unsubscribe via the World Wide Web, visit > > > ? ? ? ? https://mail.python.org/mailman/listinfo/scikit-learn > > > or, via email, send a message with subject or body 'help' to > > > ? ? ? ? scikit-learn-request(a)python.org > > > > > You can reach the person managing the list at > > > ? ? ? ? scikit-learn-owner(a)python.org > > > > > When replying, please edit your Subject line so it is more specific > > > than "Re: Contents of scikit-learn digest..." > > > > > > > Today's Topics: > > > > > ? ?1. Re: logistic regression results are not stable between > > > ? ? ? solvers (Andreas Mueller) > > > > > > > > > ---------------------------------------------------------------------- > > > > > Message: 1 > > > Date: Fri, 11 Oct 2019 15:42:58 +0200 > > > From: Andreas Mueller <t3kcit(a)gmail.com> > > > To: scikit-learn(a)python.org > > > Subject: Re: [scikit-learn] logistic regression results are not > > stable > > > ? ? ? ? between solvers > > > Message-ID: <d55949d6-3355-f892-f6b3-030edf1c7947(a)gmail.com> > > > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > > > > > > > > On 10/10/19 1:14 PM, Beno?t Presles wrote: > > > > > > Thanks for your answers. > > > > > > On my real data, I do not have so many samples. I have a bit more > > than > > > > 200 samples in total and I also would like to get some results > with > > > > unpenalized logisitic regression. > > > > What do you suggest? Should I switch to the lbfgs solver? > > > Yes. > > > > Am I sure that with this solver I will not have any convergence > > issue > > > > and always get the good result? Indeed, I did not get any > > convergence > > > > warning with saga, so I thought everything was fine. I noticed > some > > > > issues only when I decided to test several solvers. Without > > comparing > > > > the results across solvers, how to be sure that the optimisation > > goes > > > > well? Shouldn't scikit-learn warn the user somehow if it is not > > the case? > > > We should attempt to warn in the SAGA solver if it doesn't > converge. > > > That it doesn't raise a convergence warning should probably be > > > considered a bug. > > > It uses the maximum weight change as a stopping criterion right > now. > > > We could probably compute the dual objective once in the end to see > > if > > > we converged, right? Or is that not possible with SAGA? If not, we > > might > > > want to caution that no convergence warning will be raised. > > > > > > > > At last, I was using saga because I also wanted to do some > feature > > > > selection by using l1 penalty which is not supported by lbfgs... > > > You can use liblinear then. > > > > > > > > > > Best regards, > > > > Ben > > > > > > > > Le 09/10/2019 ? 23:39, Guillaume Lema?tre a ?crit?: > > > >> Ups I did not see the answer of Roman. Sorry about that. It is > > coming > > > >> back to the same conclusion :) > > > > > >> On Wed, 9 Oct 2019 at 23:37, Guillaume Lema?tre > > > >> <g.lemaitre58(a)gmail.com <mailto:g.lemaitre58@gmail.com>> wrote: > > > > > >>? ? ?Uhm actually increasing to 10000 samples solve the > convergence > > > issue. > > > >>? ? ?SAGA is not designed to work with a so small sample size > most > > > >>? ? ?probably. > > > > > >>? ? ?On Wed, 9 Oct 2019 at 23:36, Guillaume Lema?tre > > > >>? ? ?<g.lemaitre58(a)gmail.com <mailto:g.lemaitre58@gmail.com>> > > wrote: > > > > > >>? ? ? ? ?I slightly change the bench such that it uses pipeline > and > > > >>? ? ? ? ?plotted the coefficient: > > > > > >>? ? ? ? ?https://gist.github.com/glemaitre/ > > > 8fcc24bdfc7dc38ca0c09c56e26b9386 > > > > > >>? ? ? ? ?I only see one of the 10 splits where SAGA is not > > converging, > > > >>? ? ? ? ?otherwise the coefficients > > > >>? ? ? ? ?look very close (I don't attach the figure here but they > > can > > > >>? ? ? ? ?be plotted using the snippet). > > > >>? ? ? ? ?So apart from this second split, the other differences > > seems > > > >>? ? ? ? ?to be numerical instability. > > > > > >>? ? ? ? ?Where I have some concern is regarding the convergence > > rate > > > >>? ? ? ? ?of SAGA but I have no > > > >>? ? ? ? ?intuition to know if this is normal or not. > > > > > >>? ? ? ? ?On Wed, 9 Oct 2019 at 23:22, Roman Yurchak > > > >>? ? ? ? ?<rth.yurchak(a)gmail.com <mailto:rth.yurchak@gmail.com>> > > wrote: > > > > > >>? ? ? ? ? ? ?Ben, > > > > > >>? ? ? ? ? ? ?I can confirm your results with penalty='none' and > > C=1e9. > > > >>? ? ? ? ? ? ?In both cases, > > > >>? ? ? ? ? ? ?you are running a mostly unpenalized logisitic > > > >>? ? ? ? ? ? ?regression. Usually > > > >>? ? ? ? ? ? ?that's less numerically stable than with a small > > > >>? ? ? ? ? ? ?regularization, > > > >>? ? ? ? ? ? ?depending on the data collinearity. > > > > > >>? ? ? ? ? ? ?Running that same code with > > > >>? ? ? ? ? ? ?? - larger penalty ( smaller C values) > > > >>? ? ? ? ? ? ?? - or larger number of samples > > > >>? ? ? ? ? ? ?? yields for me the same coefficients (up to some > > > tolerance). > > > > > >>? ? ? ? ? ? ?You can also see that SAGA convergence is not good > by > > the > > > >>? ? ? ? ? ? ?fact that it > > > >>? ? ? ? ? ? ?needs 196000 epochs/iterations to converge. > > > > > >>? ? ? ? ? ? ?Actually, I have often seen convergence issues with > > SAG > > > >>? ? ? ? ? ? ?on small > > > >>? ? ? ? ? ? ?datasets (in unit tests), not fully sure why. > > > > > >>? ? ? ? ? ? ?-- > > > >>? ? ? ? ? ? ?Roman > > > > > >>? ? ? ? ? ? ?On 09/10/2019 22:10, serafim loukas wrote: > > > >>? ? ? ? ? ? ?> The predictions across solver are exactly the same > > when > > > >>? ? ? ? ? ? ?I run the code. > > > >>? ? ? ? ? ? ?> I am using 0.21.3 version. What is yours? > > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?> In [13]: import sklearn > > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?> In [14]: sklearn.__version__ > > > >>? ? ? ? ? ? ?> Out[14]: '0.21.3' > > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?> Serafeim > > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?>> On 9 Oct 2019, at 21:44, Beno?t Presles > > > >>? ? ? ? ? ? ?<benoit.presles(a)u-bourgogne.fr > > > >>? ? ? ? ? ? ?<mailto:benoit.presles@u-bourgogne.fr> > > > >>? ? ? ? ? ? ?>> <mailto:benoit.presles@u-bourgogne.fr > > > >>? ? ? ? ? ? ?<mailto:benoit.presles@u-bourgogne.fr>>> wrote: > > > >>? ? ? ? ? ? ?>> > > > >>? ? ? ? ? ? ?>> (y_pred_lbfgs==y_pred_saga).all() == False > > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?> _______________________________________________ > > > >>? ? ? ? ? ? ?> scikit-learn mailing list > > > >>? ? ? ? ? ? ?> scikit-learn(a)python.org <mailto: > > scikit-learn(a)python.org> > > > >>? ? ? ? ? ? ?> > > https://mail.python.org/mailman/listinfo/scikit-learn > > > >>? ? ? ? ? ? ?> > > > > > >>? ? ? ? ? ? ?_______________________________________________ > > > >>? ? ? ? ? ? ?scikit-learn mailing list > > > >>? ? ? ? ? ? ?scikit-learn(a)python.org <mailto: > > scikit-learn(a)python.org> > > > >>? ? ? ? ? ? ? > https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > > > > >>? ? ? ? ?-- > > > >>? ? ? ? ?Guillaume Lemaitre > > > >>? ? ? ? ?Scikit-learn @ Inria Foundation > > > >>? ? ? ? ?https://glemaitre.github.io/ > > > > > > > > > >>? ? ?-- > > > >>? ? ?Guillaume Lemaitre > > > >>? ? ?Scikit-learn @ Inria Foundation > > > >>? ? ?https://glemaitre.github.io/ > > > > > > > > > >> -- > > > >> Guillaume Lemaitre > > > >> Scikit-learn @ Inria Foundation > > > >> https://glemaitre.github.io/ > > > > > >> _______________________________________________ > > > >> scikit-learn mailing list > > > >> scikit-learn(a)python.org > > > >> https://mail.python.org/mailman/listinfo/scikit-learn > > > > > > _______________________________________________ > > > > scikit-learn mailing list > > > > scikit-learn(a)python.org > > > > https://mail.python.org/mailman/listinfo/scikit-learn > > > > >

4 3

Text categorization with prediction likelihood
by Daniele Nicolodi Oct. 14, 2019

Oct. 14, 2019

Hello, I don't have any formal education on predictive models, thus I hope my questions are not too naive and that the terminology I use is correct enough to make me understood. I'm trying to implement simple text categorization of phrases of a few words (the specific application is categorization of bank transaction from payee names). Following the documentation I easily implemented a solution based on the TF-IDF vectorizer and C-Support Vector machine classification. However, the problem is such that for some input phrases the classification prediction does not work that well. I have a couple of (probably very basic questions): - are my choices of algorithms the best to target this problem? Is there something else I can try to experiment with to see if I can get better results? - is there a way to obtain the prediction likelihood such that I could mark "bad" prediction for further inspection? I haven't found an (easy) way to do that in the documentation. Thank you in advance. Cheers, Dan

1 0

using numpy repeat
by Glenn Schultz Oct. 14, 2019

Oct. 14, 2019

I am tying to repeat an array 3 times using the following numpy.repeat(numpy.linspace(-.5, 3, 8), 3) axis = 0) although this repeats each element 3 times sequentially. I trying to repeat the array -.5 -0 .5 ... -5 -0 .5 any suggestions to accomplish this are appreciated. I am relatively sure I can do this with numpy I just can't put my finger on how this is done by reviewing the numpy docs. Thanks, Glenn

2 1

Announcement -- scikit-image 0.16." released
by Nelle Varoquaux Oct. 14, 2019

Oct. 14, 2019

Hi All, On behalf of the scikit-image team I am pleased to announce that scikit-image 0.16.1 has been released (0.16.0 was never released due to necessary last minute fixes). This release contains many bugs fixes and new features! Please note that we have dropped support for Python 3.5. Announcement: scikit-image 0.16.1 ================================= We're happy to announce the release of scikit-image v0.16.1! scikit-image is an image processing toolbox for SciPy that includes algorithms for segmentation, geometric transformations, color space manipulation, analysis, filtering, morphology, feature detection, and more. For more information, examples, and documentation, please visit our website: https://scikit-image.org Starting from this release, scikit-image will follow the recently introduced NumPy deprecation policy, `NEP 29 < https://github.com/numpy/numpy/blob/master/doc/neps/nep-0029-deprecation_po… >__`. Accordingly, scikit-image 0.16 drops support for Python 3.5. This release of scikit-image officially supports Python 3.6 and 3.7. Special thanks to Matthias Bussonnier for `Frappuccino <https://github.com/Carreau/frappuccino>`__, which helped us catch all API changes and nail down the APIs for new features. New Features ------------ - New `skimage.evaluate` module containing simple metrics (mse, nrme, psd) and segmentation metrics (adapted rand error, variation of information) (#4025) - n-dimensional TV-L1 optical flow algorithm for registration -- `skimage.registration.optical_flow_tvl1` (#3983) - Draw a line in an n-dimensional array -- `skimage.draw.line_nd` (#2043) - 2D Farid & Simoncelli edge filters - `skimage.filters.farid`, `skimage.filters.farid_h`, and `skimage.filters.farid_v` (#3775) - 2D majority voting filter assigning to each pixel the most commonly occurring value within its neighborhood -- `skimage.filters.majority` (#3836, #3839) - Multi-level threshold "multi-Otsu" method, a thresholding algorithm used to separate the pixels of an input image into several classes by maximizing the variances between classes -- `skimage.filters.threshold_multiotsu` (#3872, #4174) - New example data -- `skimage.data.shepp_logan_phantom`, `skimage.data.colorwheel`, `skimage.data.brick`, `skimage.data.grass`, `skimage.data.roughwall`, `skimage.data.cell` (#3958, #3966) - Compute and format image region properties as a table -- `skimage.measure.regionprops_table` (#3959) - Convert a polygon into a mask -- `skimage.draw.poly2mask` (#3971, #3977) - Visual image comparison helper `skimage.util.compare_images`, that returns an image showing the difference between two input images (#4089) - `skimage.transform.warp_polar` to remap image into polar or log-polar coordinates. (#4097) Improvements ------------ - RANSAC: new option to set initial samples selected for initialization (#2992) - Better repr and str for `skimage.transform.ProjectiveTransform` (#3525, #3967) - Better error messages and data type stability to `skimage.segmentation.relabel_sequential` (#3740) - Improved compatibility with dask arrays in some image thresholding methods (#3823) - `skimage.io.ImageCollection` can now receive lists of patterns (#3928) - Speed up `skimage.feature.peak_local_max` (#3984) - Better error message when incorrect value for keyword argument `kind` in `skimage.color.label2rgb` (#4055) - All functions from `skimage.drawing` now supports multi-channel 2D images (#4134) API Changes ----------- - Deprecated subpackage ``skimage.novice`` has been removed. - Default value of ``multichannel`` parameters has been set to False in `skimage.transform.rescale`, `skimage.transform.pyramid_reduce`, `skimage.transform.pyramid_laplacian`, `skimage.transform.pyramid_gaussian`, and `skimage.transform.pyramid_expand`. Guessing is no longer performed for 3D arrays. - Deprecated argument ``visualise`` has been removed from `skimage.feature.hog`. Use ``visualize`` instead.¨ - `skimage.transform.seam_carve` has been completely removed from the library due to licensing restrictions. - Parameter ``as_grey`` has been removed from `skimage.data.load` and `skimage.io.imread`. Use ``as_gray`` instead. - Parameter ``min_size`` has been removed from `skimage.morphology.remove_small_holes`. Use ``area_threshold`` instead. - Deprecated ``correct_mesh_orientation`` in `skimage.measure` has been removed. - `skimage.measure._regionprops` has been completely switched to using row-column coordinates. Old x-y interface is not longer available. - Default value of ``behavior`` parameter has been set to ``ndimage`` in `skimage.filters.median`. - Parameter ``flatten`` in `skimage.io.imread` has been removed in favor of ``as_gray``. - Parameters ``Hxx, Hxy, Hyy`` have been removed from `skimage.feature.corner.hessian_matrix_eigvals` in favor of ``H_elems``. - Default value of ``order`` parameter has been set to ``rc`` in `skimage.feature.hessian_matrix`. - ``skimage.util.img_as_*`` functions no longer raise precision and/or loss warnings. Bugfixes -------- - Corrected error with scales attribute in ORB.detect_and_extract (#2835) The scales attribute wasn't taking into account the mask, and thus was using an incorrect array size. - Correct for bias in Inverse Randon Transform (`skimage.transform.irandon`) (#3067) Fixed by using the Ramp filter equation in the spatial domain as described in the reference - Fix a rounding issue that caused a rotated image to have a different size than the input (`skimage.transform.rotate`) (#3173) - RANSAC uses random subsets of the original data and not bootstraps. (#3901, #3915) - Canny now produces the same output regardless of dtype (#3919) - Geometry Transforms: avoid division by zero & some degenerate cases (#3926) - Fixed float32 support in denoise_bilateral and denoise_tv_bregman (#3936) - Fixed computation of Meijering filter and avoid ZeroDivisionError (#3957) - Fixed `skimage.filters.threshold_li` to prevent being stuck on stationnary points, and thus at local minima or maxima (#3966) - Edited `skimage.exposure.rescale_intensity` to return input image instead of nans when all 0 (#4015) - Fixed `skimage.morphology.medial_axis`. A wrong indentation in Cython caused the function to not behave as intended. (#4060) - Fixed `skimage.restoration.denoise_bilateral` by correcting the padding in the gaussian filter(#4080) - Fixed `skimage.measure.find_contours` when input image contains NaN. Contours interesting NaN will be left open (#4150) - Fixed `skimage.feature.blob_log` and `skimage.feature.blob_dog` for 3D images and anisotropic data (#4162) - Fixed `skimage.exposure.adjust_gamma`, `skimage.exposure.adjust_log`, and `skimage.exposure.adjust_sigmoid` such that when provided with a 1 by 1 ndarray, it returns 1 by 1 ndarrays and not single number floats (#4169) Deprecations ------------ - Parameter ``neighbors`` in `skimage.measure.convex_hull_object` has been deprecated in favor of ``connectivity`` and will be removed in version 0.18.0. - The following functions are deprecated in favor of the `skimage.metrics` module (#4025): - `skimage.measure.compare_mse` - `skimage.measure.compare_nrmse` - `skimage.measure.compare_psnr` - `skimage.measure.compare_ssim` - The function `skimage.color.guess_spatial_dimensions` is deprecated and will be removed in 0.18 (#4031) - The argument ``bc`` in `skimage.segmentation.active_contour` is deprecated. - The function `skimage.data.load` is deprecated and will be removed in 0.18 (#4061) - The function `skimage.transform.match_histogram` is deprecated in favor of `skimage.exposure.match_histogram` (#4107) - The parameter ``neighbors`` of `skimage.morphology.convex_hull_object` is deprecated. - The `skimage.transform.randon_tranform` function will convert input image of integer type to float by default in 0.18. To preserve current behaviour, set the new argument ``preserve_range`` to True. (#4131) Documentation improvements -------------------------- - DOC: Improve the documentation of transform.resize with respect to the anti_aliasing_sigma parameter (#3911) - Fix URL for stain deconvolution reference (#3862) - Fix doc for denoise guassian (#3869) - DOC: various enhancements (cross links, gallery, ref...), mainly for corner detection (#3996) - [DOC] clarify that the inertia_tensor may be nD in documentation (#4013) - [DOC] How to test and write benchmarks (#4016) - Spellcheck @CONTRIBUTING.txt (#4008) - Spellcheck @doc/examples/segmentation/plot_watershed.py (#4009) - Spellcheck @doc/examples/segmentation/plot_thresholding.py (#4010) - Spellcheck @skimage/morphology/binary.py (#4011) - Spellcheck @skimage/morphology/extrema.py (#4012) - docs update for downscale_local_mean and N-dimensional images (#4079) - Remove fancy language from 0.15 release notes (#3827) - Documentation formatting / compilation fixes (#3838) - Remove duplicated section in INSTALL.txt. (#3876) - ENH: doc of ridge functions (#3933) - Fix docstring for Threshold Niblack (#3917) - adding docs to circle_perimeter_aa (#4155) - Update link to NumPy docstring standard in Contribution Guide (replaces #4191) (#4192) - DOC: Improve downscale_local_mean() docstring (#4180) - DOC: enhance the result display in ransac gallery example (#4109) - Gallery: use fstrings for better readability (#4110) - MNT: Document stacklevel parameter in contribution guide (#4066) - Fix minor typo (#3988) - MIN: docstring improvements in canny functions (#3920) - Minor docstring fixes for #4150 (#4184) - Fix `full` parameter description in compare_ssim (#3860) - State Bradley threshold equivalence in Niblack docstring (#3891) - Add plt.show() to example-code for consistency. (#3908) - CC0 is not equivalent to public domain. Fix the note of the horse image (#3931) - Update the joblib link in tutorial_parallelization.rst (#3943) - Fix plot_edge_filter.py references (#3946) - Add missing argument to docstring of PaintTool (#3970) - Improving documentation and tests for directional filters (#3956) - Added new thorough examples on the inner working of ``skimage.filters.threshold_li`` (#3966) - matplotlib: remove interpolation=nearest, none in our examples (#4002) - fix URL encoding for wikipedia references in filters.rank.entropy and filters.rank.shannon_entropy docstring (#4007) - Fixup integer division in examples (#4032) - Update the links the installation guide (#4118) - Gallery hough line transform (#4124) - Cross-linking between function documentation should now be much improved! (#4188) - Better documentation of the ``num_peaks`` of `skimage.feature.corner_peaks` (#4195) Other Pull Requests ------------------- - Add benchmark suite for exposure module (#3312) - Remove precision and sign loss warnings from ``skimage.util.img_as_`` (#3575) - Propose SKIPs and add mission/vision/values, governance (#3585) - Use user-installed tifffile if available (#3650) - Simplify benchmarks pinnings (#3711) - Add project_urls to setup for PyPI and other services (#3834) - Address deprecations for 0.16 release (#3841) - Followup deprecations for 0.16 (#3851) - Build and test the docs in Azure (#3873) - Pin numpydoc to pre-0.8 to fix dev docs formatting (#3893) - Change all HTTP links to HTTPS (#3896) - Skip extra deps on OSX (#3898) - Add location for Sphinx 2.0.1 search results; clean up templates (#3899) - Fix CSS styling of Sphinx 2.0.1 + numpydoc 0.9 rendered docs (#3900) - Travis CI: The sudo: tag is deprcated in Travis (#4164) - MNT Preparing the 0.16 release (#4204) - FIX generate_release_note when contributor_set contains None (#4205) - Specify that travis should use Ubuntu xenial (14.04) not trusty (16.04) (#4082) - MNT: set stack level accordingly in lab2xyz (#4067) - MNT: fixup stack level for filters ridges (#4068) - MNT: remove unused import `deprecated` from filters.thresholding (#4069) - MNT: Set stacklevel correctly in io matplotlib plugin (#4070) - MNT: set stacklevel accordingly in felzenszwalb_cython (#4071) - MNT: Set stacklevel accordingly in img_as_* (convert) (#4072) - MNT: set stacklevel accordingly in util.shape (#4073) - MNT: remove extreneous matplotlib warning (#4074) - Suppress warnings in tests for viewer (#4017) - Suppress warnings in test suite regarding measure.label (#4018) - Suppress warnings in test_rank due to type conversion (#4019) - Add todo item for imread plugin testing (#3907) - Remove matplotlib agg warning when using the sphinx gallery. (#3897) - Forward-port release notes for 0.14.4 (#4137) - Add tests for pathological arrays in threshold_li (#4143) - setup.py: Fail gracefully when NumPy is not installed (#4181) - Drop Python 3.5 support (#4102) - Force imageio reader to return NumPy arrays (#3837) - Fixing connecting to GitHub with SSH info. (#3875) - Small fix to an error message of `skimage.measure.regionprops` (#3884) - Unify skeletonize and skeletonize 3D APIs (#3904) - Add location for Sphinx 2.0.1 search results; clean up templates (#3910) - Pin numpy version forward (#3925) - Replacing pyfits with Astropy to read FITS (#3930) - Add warning for future dtype kwarg removal (#3932) - MAINT: cleanup regionprop add PYTHONOPTIMIZE=2 to travis array (#3934) - Adding complexity and new tests for filters.threshold_multiotsu (#3935) - Fixup dtype kwarg warning in certain image plugins (#3948) - don't cast integer to float before using it as integer in numpy logspace (#3949) - avoid low contrast image save in a doctest. (#3953) - MAINT: Remove unused _convert_input from filters._gaussian (#4001) - Set minimum version for imread so that it compiles from source on linux in test builds (#3960) - Cleanup plugin utilization in data.load and testsuite (#3961) - Select minimum imageio such that it is compatible with pathlib (#3969) - Remove pytest-faulthandler from test dependencies (#3987) - Fix tifffile and __array_function__ failures in our CI (#3992) - MAINT: Do not use assert in code, raise an exception instead. (#4006) - Enable packagers to disable failures on warnings. (#4021) - Fix numpy 117 rc and dask in thresholding filters (#4022) - silence r,c warnings when property does not depend on r,c (#4027) - remove warning filter, fix doc wrt r,c (#4028) - Import Iterable from collections.abc (#4033) - Import Iterable from collections.abc in vendored tifffile code (#4034) - Correction of typos after #4025 (#4036) - Rename internal function called assert_* -> check_* (#4037) - Improve import time (#4039) - Remove .meeseeksdev.yml (#4045) - Fix mpl deprecation on grid() (#4049) - Fix gallery after deprecation from #4025 (#4050) - fix mpl future deprecation normed -> density (#4053) - Add shape= to circle perimeter in hough_circle example (#4047) - Critical: address internal warnings in test suite related to metrics 4025 (#4063) - Use functools instead of a real function for the internal warn function (#4062) - Test rank capture warnings in threadsafe manner (#4064) - Make use of FFTs more consistent across the library (#4084) - Fixup region props test (#4099) - Turn single backquotes to double backquotes in filters (#4127) - Refactor radon transform module (#4136) - Fix broken import of rgb2gray in benchmark suite (#4176) - Fix doc building issues with SKIPs (#4182) - Remove several __future__ imports (#4198) - Restore deprecated coordinates arg to regionprops (#4144) - Refactor/optimize threshold_multiotsu (#4167) - Remove Python2-specific code (#4170) - `view_as_windows` incorrectly assumes that a contiguous array is needed (#4171) - Handle case in which NamedTemporaryFile fails (#4172) - Fix incorrect resolution date on SKIP1 (#4183) - API updates before 0.16 (#4187) - Fix conversion to float32 dtype (#4193) Contributors to this release ---------------------------- - Abhishek Arya - Alexandre de Siqueira - Alexis Mignon - Anthony Carapetis - Bastian Eichenberger - Bharat Raghunathan - Christian Clauss - Clement Ng - David Breuer - David Haberthür - Dominik Kutra - Dominik Straub - Egor Panfilov - Emmanuelle Gouillart - Etienne Landuré - François Boulogne - Genevieve Buckley - Gregory R. Lee - Hadrien Mary - Hamdi Sahloul - Holly Gibbs - Huang-Wei Chang - i3v (i3v) - Jarrod Millman - Jirka Borovec - Johan Jeppsson - Johannes Schönberger - Jon Crall - Josh Warner - Juan Nunez-Iglesias - Kaligule (Kaligule) - kczimm (kczimm) - Lars Grueter - Shachar Ben Harim - Luis F. de Figueiredo - Mark Harfouche - Mars Huang - Dave Mellert - Nelle Varoquaux - Ollin Boer Bohan - Patrick J Zager - Riadh Fezzani - Ryan Avery - Srinath Kailasa - Stefan van der Walt - Stuart Berg - Uwe Schmidt Reviewers for this release -------------------------- - Alexandre de Siqueira - Anthony Carapetis - Bastian Eichenberger - Clement Ng - David Breuer - Egor Panfilov - Emmanuelle Gouillart - Etienne Landuré - François Boulogne - Genevieve Buckley - Gregory R. Lee - Hadrien Mary - Hamdi Sahloul - Holly Gibbs - Jarrod Millman - Jirka Borovec - Johan Jeppsson - Johannes Schönberger - Jon Crall - Josh Warner - jrmarsha - Juan Nunez-Iglesias - kczimm - Lars Grueter - leGIT-bot - Mark Harfouche - Mars Huang - Dave Mellert - Paul Müller - Phil Starkey - Ralf Gommers - Riadh Fezzani - Ryan Avery - Sebastian Berg - Stefan van der Walt - Uwe Schmidt

1 0

Re: [scikit-learn] scikit-learn Digest, Vol 43, Issue 24
by Mike Smith Oct. 12, 2019

Oct. 12, 2019

Gael, simply because you're not able to or willing to answer the question doesn't mean there is no practical answer for it. "...> How much hardware is a practical expectation for running the best > scikit models and getting the best results? This is too vague a question for which there is no answer. Ga?l " On Sat, Oct 12, 2019 at 9:03 AM <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: Is scikit-learn implying neural nets are the best > regressor? (Gael Varoquaux) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 11 Oct 2019 13:34:33 -0400 > From: Gael Varoquaux <gael.varoquaux(a)normalesup.org> > To: Scikit-learn mailing list <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] Is scikit-learn implying neural nets are > the best regressor? > Message-ID: <20191011173433.bbywiqnwjjpvsi4r(a)phare.normalesup.org> > Content-Type: text/plain; charset=iso-8859-1 > > On Fri, Oct 11, 2019 at 10:10:32AM -0700, Mike Smith wrote: > > In other words, according to that arrangement, is scikit-learn implying > that > > section 1.17 is the best regressor out of the listed, 1.1 to 1.17? > > No. > > First they are not ordered in order of complexity (Naive Bayes is > arguably simpler than Gaussian Processes). Second complexity does not > imply better prediction. > > > If I should expect good results on a pc, scikit says that needing gpu > power is > > obsolete, since certain scikit models perform better (than ml designed > for gpu) > > that are not designed for gpu, for that reason. Is this true? > > Where do you see this written? I think that you are looking for overly > simple stories that you are not true. > > > How much hardware is a practical expectation for running the best > > scikit models and getting the best results? > > This is too vague a question for which there is no answer. > > Ga?l > > > On Fri, Oct 11, 2019 at 9:02 AM <scikit-learn-request(a)python.org> wrote: > > > Send scikit-learn mailing list submissions to > > ? ? ? ? scikit-learn(a)python.org > > > To subscribe or unsubscribe via the World Wide Web, visit > > ? ? ? ? https://mail.python.org/mailman/listinfo/scikit-learn > > or, via email, send a message with subject or body 'help' to > > ? ? ? ? scikit-learn-request(a)python.org > > > You can reach the person managing the list at > > ? ? ? ? scikit-learn-owner(a)python.org > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of scikit-learn digest..." > > > > Today's Topics: > > > ? ?1. Re: logistic regression results are not stable between > > ? ? ? solvers (Andreas Mueller) > > > > > ---------------------------------------------------------------------- > > > Message: 1 > > Date: Fri, 11 Oct 2019 15:42:58 +0200 > > From: Andreas Mueller <t3kcit(a)gmail.com> > > To: scikit-learn(a)python.org > > Subject: Re: [scikit-learn] logistic regression results are not > stable > > ? ? ? ? between solvers > > Message-ID: <d55949d6-3355-f892-f6b3-030edf1c7947(a)gmail.com> > > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > > > > On 10/10/19 1:14 PM, Beno?t Presles wrote: > > > > Thanks for your answers. > > > > On my real data, I do not have so many samples. I have a bit more > than > > > 200 samples in total and I also would like to get some results with > > > unpenalized logisitic regression. > > > What do you suggest? Should I switch to the lbfgs solver? > > Yes. > > > Am I sure that with this solver I will not have any convergence > issue > > > and always get the good result? Indeed, I did not get any > convergence > > > warning with saga, so I thought everything was fine. I noticed some > > > issues only when I decided to test several solvers. Without > comparing > > > the results across solvers, how to be sure that the optimisation > goes > > > well? Shouldn't scikit-learn warn the user somehow if it is not > the case? > > We should attempt to warn in the SAGA solver if it doesn't converge. > > That it doesn't raise a convergence warning should probably be > > considered a bug. > > It uses the maximum weight change as a stopping criterion right now. > > We could probably compute the dual objective once in the end to see > if > > we converged, right? Or is that not possible with SAGA? If not, we > might > > want to caution that no convergence warning will be raised. > > > > > At last, I was using saga because I also wanted to do some feature > > > selection by using l1 penalty which is not supported by lbfgs... > > You can use liblinear then. > > > > > > Best regards, > > > Ben > > > > > Le 09/10/2019 ? 23:39, Guillaume Lema?tre a ?crit?: > > >> Ups I did not see the answer of Roman. Sorry about that. It is > coming > > >> back to the same conclusion :) > > > >> On Wed, 9 Oct 2019 at 23:37, Guillaume Lema?tre > > >> <g.lemaitre58(a)gmail.com <mailto:g.lemaitre58@gmail.com>> wrote: > > > >>? ? ?Uhm actually increasing to 10000 samples solve the convergence > > issue. > > >>? ? ?SAGA is not designed to work with a so small sample size most > > >>? ? ?probably. > > > >>? ? ?On Wed, 9 Oct 2019 at 23:36, Guillaume Lema?tre > > >>? ? ?<g.lemaitre58(a)gmail.com <mailto:g.lemaitre58@gmail.com>> > wrote: > > > >>? ? ? ? ?I slightly change the bench such that it uses pipeline and > > >>? ? ? ? ?plotted the coefficient: > > > >>? ? ? ? ?https://gist.github.com/glemaitre/ > > 8fcc24bdfc7dc38ca0c09c56e26b9386 > > > >>? ? ? ? ?I only see one of the 10 splits where SAGA is not > converging, > > >>? ? ? ? ?otherwise the coefficients > > >>? ? ? ? ?look very close (I don't attach the figure here but they > can > > >>? ? ? ? ?be plotted using the snippet). > > >>? ? ? ? ?So apart from this second split, the other differences > seems > > >>? ? ? ? ?to be numerical instability. > > > >>? ? ? ? ?Where I have some concern is regarding the convergence > rate > > >>? ? ? ? ?of SAGA but I have no > > >>? ? ? ? ?intuition to know if this is normal or not. > > > >>? ? ? ? ?On Wed, 9 Oct 2019 at 23:22, Roman Yurchak > > >>? ? ? ? ?<rth.yurchak(a)gmail.com <mailto:rth.yurchak@gmail.com>> > wrote: > > > >>? ? ? ? ? ? ?Ben, > > > >>? ? ? ? ? ? ?I can confirm your results with penalty='none' and > C=1e9. > > >>? ? ? ? ? ? ?In both cases, > > >>? ? ? ? ? ? ?you are running a mostly unpenalized logisitic > > >>? ? ? ? ? ? ?regression. Usually > > >>? ? ? ? ? ? ?that's less numerically stable than with a small > > >>? ? ? ? ? ? ?regularization, > > >>? ? ? ? ? ? ?depending on the data collinearity. > > > >>? ? ? ? ? ? ?Running that same code with > > >>? ? ? ? ? ? ?? - larger penalty ( smaller C values) > > >>? ? ? ? ? ? ?? - or larger number of samples > > >>? ? ? ? ? ? ?? yields for me the same coefficients (up to some > > tolerance). > > > >>? ? ? ? ? ? ?You can also see that SAGA convergence is not good by > the > > >>? ? ? ? ? ? ?fact that it > > >>? ? ? ? ? ? ?needs 196000 epochs/iterations to converge. > > > >>? ? ? ? ? ? ?Actually, I have often seen convergence issues with > SAG > > >>? ? ? ? ? ? ?on small > > >>? ? ? ? ? ? ?datasets (in unit tests), not fully sure why. > > > >>? ? ? ? ? ? ?-- > > >>? ? ? ? ? ? ?Roman > > > >>? ? ? ? ? ? ?On 09/10/2019 22:10, serafim loukas wrote: > > >>? ? ? ? ? ? ?> The predictions across solver are exactly the same > when > > >>? ? ? ? ? ? ?I run the code. > > >>? ? ? ? ? ? ?> I am using 0.21.3 version. What is yours? > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> In [13]: import sklearn > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> In [14]: sklearn.__version__ > > >>? ? ? ? ? ? ?> Out[14]: '0.21.3' > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> Serafeim > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?>> On 9 Oct 2019, at 21:44, Beno?t Presles > > >>? ? ? ? ? ? ?<benoit.presles(a)u-bourgogne.fr > > >>? ? ? ? ? ? ?<mailto:benoit.presles@u-bourgogne.fr> > > >>? ? ? ? ? ? ?>> <mailto:benoit.presles@u-bourgogne.fr > > >>? ? ? ? ? ? ?<mailto:benoit.presles@u-bourgogne.fr>>> wrote: > > >>? ? ? ? ? ? ?>> > > >>? ? ? ? ? ? ?>> (y_pred_lbfgs==y_pred_saga).all() == False > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> _______________________________________________ > > >>? ? ? ? ? ? ?> scikit-learn mailing list > > >>? ? ? ? ? ? ?> scikit-learn(a)python.org <mailto: > scikit-learn(a)python.org> > > >>? ? ? ? ? ? ?> > https://mail.python.org/mailman/listinfo/scikit-learn > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?_______________________________________________ > > >>? ? ? ? ? ? ?scikit-learn mailing list > > >>? ? ? ? ? ? ?scikit-learn(a)python.org <mailto: > scikit-learn(a)python.org> > > >>? ? ? ? ? ? ?https://mail.python.org/mailman/listinfo/scikit-learn > > > > > >>? ? ? ? ?-- > > >>? ? ? ? ?Guillaume Lemaitre > > >>? ? ? ? ?Scikit-learn @ Inria Foundation > > >>? ? ? ? ?https://glemaitre.github.io/ > > > > > >>? ? ?-- > > >>? ? ?Guillaume Lemaitre > > >>? ? ?Scikit-learn @ Inria Foundation > > >>? ? ?https://glemaitre.github.io/ > > > > > >> -- > > >> Guillaume Lemaitre > > >> Scikit-learn @ Inria Foundation > > >> https://glemaitre.github.io/ > > > >> _______________________________________________ > > >> scikit-learn mailing list > > >> scikit-learn(a)python.org > > >> https://mail.python.org/mailman/listinfo/scikit-learn > > > > _______________________________________________ > > > scikit-learn mailing list > > > scikit-learn(a)python.org > > > https://mail.python.org/mailman/listinfo/scikit-learn > > >

1 0

Re: [scikit-learn] scikit-learn Digest, Vol 43, Issue 24
by Mike Smith Oct. 12, 2019

Oct. 12, 2019

"... > If I should expect good results on a pc, scikit says that needing gpu power is > obsolete, since certain scikit models perform better (than ml designed for gpu) > that are not designed for gpu, for that reason. Is this true?" Where do you see this written? I think that you are looking for overly simple stories that you are not true." Gael, see the below from the scikit-learn FAQ. You can also find this yourself at the main FAQ: [image: 2019-10-12 14_00_05-Frequently Asked Questions — scikit-learn 0.21.3 documentation.png] On Sat, Oct 12, 2019 at 9:03 AM <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: Is scikit-learn implying neural nets are the best > regressor? (Gael Varoquaux) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 11 Oct 2019 13:34:33 -0400 > From: Gael Varoquaux <gael.varoquaux(a)normalesup.org> > To: Scikit-learn mailing list <scikit-learn(a)python.org> > Subject: Re: [scikit-learn] Is scikit-learn implying neural nets are > the best regressor? > Message-ID: <20191011173433.bbywiqnwjjpvsi4r(a)phare.normalesup.org> > Content-Type: text/plain; charset=iso-8859-1 > > On Fri, Oct 11, 2019 at 10:10:32AM -0700, Mike Smith wrote: > > In other words, according to that arrangement, is scikit-learn implying > that > > section 1.17 is the best regressor out of the listed, 1.1 to 1.17? > > No. > > First they are not ordered in order of complexity (Naive Bayes is > arguably simpler than Gaussian Processes). Second complexity does not > imply better prediction. > > > If I should expect good results on a pc, scikit says that needing gpu > power is > > obsolete, since certain scikit models perform better (than ml designed > for gpu) > > that are not designed for gpu, for that reason. Is this true? > > Where do you see this written? I think that you are looking for overly > simple stories that you are not true. > > > How much hardware is a practical expectation for running the best > > scikit models and getting the best results? > > This is too vague a question for which there is no answer. > > Ga?l > > > On Fri, Oct 11, 2019 at 9:02 AM <scikit-learn-request(a)python.org> wrote: > > > Send scikit-learn mailing list submissions to > > ? ? ? ? scikit-learn(a)python.org > > > To subscribe or unsubscribe via the World Wide Web, visit > > ? ? ? ? https://mail.python.org/mailman/listinfo/scikit-learn > > or, via email, send a message with subject or body 'help' to > > ? ? ? ? scikit-learn-request(a)python.org > > > You can reach the person managing the list at > > ? ? ? ? scikit-learn-owner(a)python.org > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of scikit-learn digest..." > > > > Today's Topics: > > > ? ?1. Re: logistic regression results are not stable between > > ? ? ? solvers (Andreas Mueller) > > > > > ---------------------------------------------------------------------- > > > Message: 1 > > Date: Fri, 11 Oct 2019 15:42:58 +0200 > > From: Andreas Mueller <t3kcit(a)gmail.com> > > To: scikit-learn(a)python.org > > Subject: Re: [scikit-learn] logistic regression results are not > stable > > ? ? ? ? between solvers > > Message-ID: <d55949d6-3355-f892-f6b3-030edf1c7947(a)gmail.com> > > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > > > > On 10/10/19 1:14 PM, Beno?t Presles wrote: > > > > Thanks for your answers. > > > > On my real data, I do not have so many samples. I have a bit more > than > > > 200 samples in total and I also would like to get some results with > > > unpenalized logisitic regression. > > > What do you suggest? Should I switch to the lbfgs solver? > > Yes. > > > Am I sure that with this solver I will not have any convergence > issue > > > and always get the good result? Indeed, I did not get any > convergence > > > warning with saga, so I thought everything was fine. I noticed some > > > issues only when I decided to test several solvers. Without > comparing > > > the results across solvers, how to be sure that the optimisation > goes > > > well? Shouldn't scikit-learn warn the user somehow if it is not > the case? > > We should attempt to warn in the SAGA solver if it doesn't converge. > > That it doesn't raise a convergence warning should probably be > > considered a bug. > > It uses the maximum weight change as a stopping criterion right now. > > We could probably compute the dual objective once in the end to see > if > > we converged, right? Or is that not possible with SAGA? If not, we > might > > want to caution that no convergence warning will be raised. > > > > > At last, I was using saga because I also wanted to do some feature > > > selection by using l1 penalty which is not supported by lbfgs... > > You can use liblinear then. > > > > > > Best regards, > > > Ben > > > > > Le 09/10/2019 ? 23:39, Guillaume Lema?tre a ?crit?: > > >> Ups I did not see the answer of Roman. Sorry about that. It is > coming > > >> back to the same conclusion :) > > > >> On Wed, 9 Oct 2019 at 23:37, Guillaume Lema?tre > > >> <g.lemaitre58(a)gmail.com <mailto:g.lemaitre58@gmail.com>> wrote: > > > >>? ? ?Uhm actually increasing to 10000 samples solve the convergence > > issue. > > >>? ? ?SAGA is not designed to work with a so small sample size most > > >>? ? ?probably. > > > >>? ? ?On Wed, 9 Oct 2019 at 23:36, Guillaume Lema?tre > > >>? ? ?<g.lemaitre58(a)gmail.com <mailto:g.lemaitre58@gmail.com>> > wrote: > > > >>? ? ? ? ?I slightly change the bench such that it uses pipeline and > > >>? ? ? ? ?plotted the coefficient: > > > >>? ? ? ? ?https://gist.github.com/glemaitre/ > > 8fcc24bdfc7dc38ca0c09c56e26b9386 > > > >>? ? ? ? ?I only see one of the 10 splits where SAGA is not > converging, > > >>? ? ? ? ?otherwise the coefficients > > >>? ? ? ? ?look very close (I don't attach the figure here but they > can > > >>? ? ? ? ?be plotted using the snippet). > > >>? ? ? ? ?So apart from this second split, the other differences > seems > > >>? ? ? ? ?to be numerical instability. > > > >>? ? ? ? ?Where I have some concern is regarding the convergence > rate > > >>? ? ? ? ?of SAGA but I have no > > >>? ? ? ? ?intuition to know if this is normal or not. > > > >>? ? ? ? ?On Wed, 9 Oct 2019 at 23:22, Roman Yurchak > > >>? ? ? ? ?<rth.yurchak(a)gmail.com <mailto:rth.yurchak@gmail.com>> > wrote: > > > >>? ? ? ? ? ? ?Ben, > > > >>? ? ? ? ? ? ?I can confirm your results with penalty='none' and > C=1e9. > > >>? ? ? ? ? ? ?In both cases, > > >>? ? ? ? ? ? ?you are running a mostly unpenalized logisitic > > >>? ? ? ? ? ? ?regression. Usually > > >>? ? ? ? ? ? ?that's less numerically stable than with a small > > >>? ? ? ? ? ? ?regularization, > > >>? ? ? ? ? ? ?depending on the data collinearity. > > > >>? ? ? ? ? ? ?Running that same code with > > >>? ? ? ? ? ? ?? - larger penalty ( smaller C values) > > >>? ? ? ? ? ? ?? - or larger number of samples > > >>? ? ? ? ? ? ?? yields for me the same coefficients (up to some > > tolerance). > > > >>? ? ? ? ? ? ?You can also see that SAGA convergence is not good by > the > > >>? ? ? ? ? ? ?fact that it > > >>? ? ? ? ? ? ?needs 196000 epochs/iterations to converge. > > > >>? ? ? ? ? ? ?Actually, I have often seen convergence issues with > SAG > > >>? ? ? ? ? ? ?on small > > >>? ? ? ? ? ? ?datasets (in unit tests), not fully sure why. > > > >>? ? ? ? ? ? ?-- > > >>? ? ? ? ? ? ?Roman > > > >>? ? ? ? ? ? ?On 09/10/2019 22:10, serafim loukas wrote: > > >>? ? ? ? ? ? ?> The predictions across solver are exactly the same > when > > >>? ? ? ? ? ? ?I run the code. > > >>? ? ? ? ? ? ?> I am using 0.21.3 version. What is yours? > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> In [13]: import sklearn > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> In [14]: sklearn.__version__ > > >>? ? ? ? ? ? ?> Out[14]: '0.21.3' > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> Serafeim > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?>> On 9 Oct 2019, at 21:44, Beno?t Presles > > >>? ? ? ? ? ? ?<benoit.presles(a)u-bourgogne.fr > > >>? ? ? ? ? ? ?<mailto:benoit.presles@u-bourgogne.fr> > > >>? ? ? ? ? ? ?>> <mailto:benoit.presles@u-bourgogne.fr > > >>? ? ? ? ? ? ?<mailto:benoit.presles@u-bourgogne.fr>>> wrote: > > >>? ? ? ? ? ? ?>> > > >>? ? ? ? ? ? ?>> (y_pred_lbfgs==y_pred_saga).all() == False > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> > > >>? ? ? ? ? ? ?> _______________________________________________ > > >>? ? ? ? ? ? ?> scikit-learn mailing list > > >>? ? ? ? ? ? ?> scikit-learn(a)python.org <mailto: > scikit-learn(a)python.org> > > >>? ? ? ? ? ? ?> > https://mail.python.org/mailman/listinfo/scikit-learn > > >>? ? ? ? ? ? ?> > > > >>? ? ? ? ? ? ?_______________________________________________ > > >>? ? ? ? ? ? ?scikit-learn mailing list > > >>? ? ? ? ? ? ?scikit-learn(a)python.org <mailto: > scikit-learn(a)python.org> > > >>? ? ? ? ? ? ?https://mail.python.org/mailman/listinfo/scikit-learn > > > > > >>? ? ? ? ?-- > > >>? ? ? ? ?Guillaume Lemaitre > > >>? ? ? ? ?Scikit-learn @ Inria Foundation > > >>? ? ? ? ?https://glemaitre.github.io/ > > > > > >>? ? ?-- > > >>? ? ?Guillaume Lemaitre > > >>? ? ?Scikit-learn @ Inria Foundation > > >>? ? ?https://glemaitre.github.io/ > > > > > >> -- > > >> Guillaume Lemaitre > > >> Scikit-learn @ Inria Foundation > > >> https://glemaitre.github.io/ > > > >> _______________________________________________ > > >> scikit-learn mailing list > > >> scikit-learn(a)python.org > > >> https://mail.python.org/mailman/listinfo/scikit-learn > > > > _______________________________________________ > > > scikit-learn mailing list > > > scikit-learn(a)python.org > > > https://mail.python.org/mailman/listinfo/scikit-learn > > >

1 0

Is scikit-learn implying neural nets are the best regressor?
by Mike Smith Oct. 11, 2019

Oct. 11, 2019

In other words, according to that arrangement, is scikit-learn implying that section 1.17 is the best regressor out of the listed, 1.1 to 1.17? If yes, I'd like to know if i can run the model on standard pc cpu ram and should still expect good results or do I need cloud hardware? If I should expect good results on a pc, scikit says that needing gpu power is obsolete, since certain scikit models perform better (than ml designed for gpu) that are not designed for gpu, for that reason. Is this true? How much hardware is a practical expectation for running the best scikit models and getting the best results? On Fri, Oct 11, 2019 at 9:02 AM <scikit-learn-request(a)python.org> wrote: > Send scikit-learn mailing list submissions to > scikit-learn(a)python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scikit-learn > or, via email, send a message with subject or body 'help' to > scikit-learn-request(a)python.org > > You can reach the person managing the list at > scikit-learn-owner(a)python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of scikit-learn digest..." > > > Today's Topics: > > 1. Re: logistic regression results are not stable between > solvers (Andreas Mueller) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 11 Oct 2019 15:42:58 +0200 > From: Andreas Mueller <t3kcit(a)gmail.com> > To: scikit-learn(a)python.org > Subject: Re: [scikit-learn] logistic regression results are not stable > between solvers > Message-ID: <d55949d6-3355-f892-f6b3-030edf1c7947(a)gmail.com> > Content-Type: text/plain; charset="utf-8"; Format="flowed" > > > > On 10/10/19 1:14 PM, Beno?t Presles wrote: > > > > Thanks for your answers. > > > > On my real data, I do not have so many samples. I have a bit more than > > 200 samples in total and I also would like to get some results with > > unpenalized logisitic regression. > > What do you suggest? Should I switch to the lbfgs solver? > Yes. > > Am I sure that with this solver I will not have any convergence issue > > and always get the good result? Indeed, I did not get any convergence > > warning with saga, so I thought everything was fine. I noticed some > > issues only when I decided to test several solvers. Without comparing > > the results across solvers, how to be sure that the optimisation goes > > well? Shouldn't scikit-learn warn the user somehow if it is not the case? > We should attempt to warn in the SAGA solver if it doesn't converge. > That it doesn't raise a convergence warning should probably be > considered a bug. > It uses the maximum weight change as a stopping criterion right now. > We could probably compute the dual objective once in the end to see if > we converged, right? Or is that not possible with SAGA? If not, we might > want to caution that no convergence warning will be raised. > > > > > At last, I was using saga because I also wanted to do some feature > > selection by using l1 penalty which is not supported by lbfgs... > You can use liblinear then. > > > > > > Best regards, > > Ben > > > > > > Le 09/10/2019 ? 23:39, Guillaume Lema?tre a ?crit?: > >> Ups I did not see the answer of Roman. Sorry about that. It is coming > >> back to the same conclusion :) > >> > >> On Wed, 9 Oct 2019 at 23:37, Guillaume Lema?tre > >> <g.lemaitre58(a)gmail.com <mailto:g.lemaitre58@gmail.com>> wrote: > >> > >> Uhm actually increasing to 10000 samples solve the convergence > issue. > >> SAGA is not designed to work with a so small sample size most > >> probably. > >> > >> On Wed, 9 Oct 2019 at 23:36, Guillaume Lema?tre > >> <g.lemaitre58(a)gmail.com <mailto:g.lemaitre58@gmail.com>> wrote: > >> > >> I slightly change the bench such that it uses pipeline and > >> plotted the coefficient: > >> > >> > https://gist.github.com/glemaitre/8fcc24bdfc7dc38ca0c09c56e26b9386 > >> > >> I only see one of the 10 splits where SAGA is not converging, > >> otherwise the coefficients > >> look very close (I don't attach the figure here but they can > >> be plotted using the snippet). > >> So apart from this second split, the other differences seems > >> to be numerical instability. > >> > >> Where I have some concern is regarding the convergence rate > >> of SAGA but I have no > >> intuition to know if this is normal or not. > >> > >> On Wed, 9 Oct 2019 at 23:22, Roman Yurchak > >> <rth.yurchak(a)gmail.com <mailto:rth.yurchak@gmail.com>> wrote: > >> > >> Ben, > >> > >> I can confirm your results with penalty='none' and C=1e9. > >> In both cases, > >> you are running a mostly unpenalized logisitic > >> regression. Usually > >> that's less numerically stable than with a small > >> regularization, > >> depending on the data collinearity. > >> > >> Running that same code with > >> ? - larger penalty ( smaller C values) > >> ? - or larger number of samples > >> ? yields for me the same coefficients (up to some > tolerance). > >> > >> You can also see that SAGA convergence is not good by the > >> fact that it > >> needs 196000 epochs/iterations to converge. > >> > >> Actually, I have often seen convergence issues with SAG > >> on small > >> datasets (in unit tests), not fully sure why. > >> > >> -- > >> Roman > >> > >> On 09/10/2019 22:10, serafim loukas wrote: > >> > The predictions across solver are exactly the same when > >> I run the code. > >> > I am using 0.21.3 version. What is yours? > >> > > >> > > >> > In [13]: import sklearn > >> > > >> > In [14]: sklearn.__version__ > >> > Out[14]: '0.21.3' > >> > > >> > > >> > Serafeim > >> > > >> > > >> > > >> >> On 9 Oct 2019, at 21:44, Beno?t Presles > >> <benoit.presles(a)u-bourgogne.fr > >> <mailto:benoit.presles@u-bourgogne.fr> > >> >> <mailto:benoit.presles@u-bourgogne.fr > >> <mailto:benoit.presles@u-bourgogne.fr>>> wrote: > >> >> > >> >> (y_pred_lbfgs==y_pred_saga).all() == False > >> > > >> > > >> > _______________________________________________ > >> > scikit-learn mailing list > >> > scikit-learn(a)python.org <mailto:scikit-learn@python.org> > >> > https://mail.python.org/mailman/listinfo/scikit-learn > >> > > >> > >> _______________________________________________ > >> scikit-learn mailing list > >> scikit-learn(a)python.org <mailto:scikit-learn@python.org> > >> https://mail.python.org/mailman/listinfo/scikit-learn > >> > >> > >> > >> -- > >> Guillaume Lemaitre > >> Scikit-learn @ Inria Foundation > >> https://glemaitre.github.io/ > >> > >> > >> > >> -- > >> Guillaume Lemaitre > >> Scikit-learn @ Inria Foundation > >> https://glemaitre.github.io/ > >> > >> > >> > >> -- > >> Guillaume Lemaitre > >> Scikit-learn @ Inria Foundation > >> https://glemaitre.github.io/ > >> > >> _______________________________________________ > >> scikit-learn mailing list > >> scikit-learn(a)python.org > >> https://mail.python.org/mailman/listinfo/scikit-learn > > > > _______________________________________________ > > scikit-learn mailing list > > scikit-learn(a)python.org > > https://mail.python.org/mailman/listinfo/scikit-learn > >

2 1