Logistic regression with elastic net regularization
Is there an implementation of logistic regression with elastic net regularization in scikit? (or pointers on implementing this - its seems non-convex and so you might expect poor behavior with some optimizers) - Stuart
Hi Stuart Take a look at this issue: https://github.com/scikit-learn/scikit-learn/issues/2968 On Mon, Mar 13, 2017 at 9:57 AM, Stuart Reynolds <stuart@stuartreynolds.net> wrote:
Is there an implementation of logistic regression with elastic net regularization in scikit? (or pointers on implementing this - its seems non-convex and so you might expect poor behavior with some optimizers)
- Stuart
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
Perfect. Thanks -- will give it a go. On Mon, Mar 13, 2017 at 10:04 AM, Jacob Schreiber <jmschreiber91@gmail.com> wrote:
Hi Stuart
Take a look at this issue: https://github.com/scikit-learn/scikit-learn/ issues/2968
On Mon, Mar 13, 2017 at 9:57 AM, Stuart Reynolds < stuart@stuartreynolds.net> wrote:
Is there an implementation of logistic regression with elastic net regularization in scikit? (or pointers on implementing this - its seems non-convex and so you might expect poor behavior with some optimizers)
- Stuart
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
Recently, there are some issues/PRs tackling the topic: https://github.com/scikit-learn/scikit-learn/issues/8288 https://github.com/scikit-learn/scikit-learn/issues/8446
Hi, Stuart, I think the only way to do that right now would be through the SGD classifier, e.g., sklearn.linear_model.SGDClassifier(loss='log', penalty='elasticnet' …) Best, Sebastian
On Mar 13, 2017, at 12:57 PM, Stuart Reynolds <stuart@stuartreynolds.net> wrote:
Is there an implementation of logistic regression with elastic net regularization in scikit? (or pointers on implementing this - its seems non-convex and so you might expect poor behavior with some optimizers)
- Stuart _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
Many thanks. On Mon, Mar 13, 2017 at 10:08 AM, Sebastian Raschka <se.raschka@gmail.com> wrote:
Hi, Stuart, I think the only way to do that right now would be through the SGD classifier, e.g.,
sklearn.linear_model.SGDClassifier(loss='log', penalty='elasticnet' …)
Best, Sebastian
On Mar 13, 2017, at 12:57 PM, Stuart Reynolds <stuart@stuartreynolds.net> wrote:
Is there an implementation of logistic regression with elastic net regularization in scikit? (or pointers on implementing this - its seems non-convex and so you might expect poor behavior with some optimizers)
- Stuart _______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
_______________________________________________ scikit-learn mailing list scikit-learn@python.org https://mail.python.org/mailman/listinfo/scikit-learn
Note that SGD is not very good at optimizing finely with a non-smooth penalty (e.g. l1 or elasticnet). The future SAGA solver is going to be much better at finding the optimal sparsity support (although this support is not guaranteed to be stable across re-sampling of the training set if the training set is small). -- Olivier
From a generalization point of view (test accuracy), the optimal sparsity support should not matter much though, but it can be helpful to find a the optimally sparsest solution for either computational constraints (smaller models with a lower prediction latency) and interpretation of the weights (domain specific).
-- Olivier
participants (5)
-
Guillaume Lemaitre -
Jacob Schreiber -
Olivier Grisel -
Sebastian Raschka -
Stuart Reynolds