[scikit-learn] RFE with logistic regression

[Andreas Mueller]

Can you share your data or reproduce with synthetic data?

On 07/24/2018 02:43 PM, Benoît Presles wrote:
> I did the same tests as before adding random_state=0 and:
>
> 1. I have got the same problem as before, i.e. when I execute the RFE 
> multiple times I don't get the same ranking each time.
>
> 2. When I change the solver to 'sag' or 'saga' 
> (LogisticRegression(C=1e9, verbose=1, max_iter=10000, 
> fit_intercept=False, random_state=0, solver='sag')), it
> seems that I get the same results at each run but the ranking is not 
> the same between these two solvers.
>
> 3. With C=1, it seems that I have the same results at each run for all 
> solvers (liblinear, sag and saga), however the ranking is not the same 
> between the solvers.
>
> Thanks for your help,
> Ben
>
>
> PS1: I checked and n_iter_ seems to be always lower than max_iter.
> PS2: my data is scaled, I am using "StandardScaler".
>
>
>
> Le 24/07/2018 à 20:33, Andreas Mueller a écrit :
>>
>>
>> On 07/24/2018 02:07 PM, Benoît Presles wrote:
>>> I did the same tests as before adding fit_intercept=False and:
>>>
>>> 1. I have got the same problem as before, i.e. when I execute the 
>>> RFE multiple times I don't get the same ranking each time.
>>>
>>> 2. When I change the solver to 'sag' 
>>> (classifier_RFE=LogisticRegression(C=1e9, verbose=1, max_iter=10000, 
>>> fit_intercept=False, solver='sag')), it seems that I get the same 
>>> ranking at each run. This is not the case with the 'saga' solver.
>>> The ranking is not the same between the solvers.
>>>
>>> 3. With C=1, it seems that I have the same results at each run for 
>>> all solvers (liblinear, sag and saga), however the ranking is not 
>>> the same between the solvers.
>>>
>>>
>>> How can I get reproducible and consistent results?
>>>
>> Did you scale your data? If not, saga and sag will basically fail.
>> _______________________________________________
>> scikit-learn mailing list
>> scikit-learn at python.org
>> https://mail.python.org/mailman/listinfo/scikit-learn
>
> _______________________________________________
> scikit-learn mailing list
> scikit-learn at python.org
> https://mail.python.org/mailman/listinfo/scikit-learn