[scikit-learn] Implement Catalyst-SVRG optimizer for linear models?

Krishna Pillutla krish.pvkrishna at gmail.com
Thu Feb 22 12:08:51 EST 2018

Hello all,

*TL;DR*: I'd like to implement Catalyst-SVRG
<https://arxiv.org/pdf/1712.05654.pdf>, an accelerated optimization
algorithm for sklearn (or scikit-learn-contrib/lightning, if it is more
appropriate). Any feedback?

*Long version*:
I've been playing around with Catalyst-SVRG
<https://arxiv.org/pdf/1712.05654.pdf>, an accelerated stochastic variance
reduced optimization algorithm for my research. I've found in my experience
and in the experiments section of the attached paper that this algorithm
does lead to faster optimization than vanilla (un-accerelated) SVRG
which itself is much faster than SGD and on roughly the same footing as
SAG/SAGA. Moreover, the per-iteration computational complexity of this
algorithm practically matches that of SVRG.

I was wondering whether it would be beneficial to the community if I
implemented this algorithm in sklearn/linear_model or perhaps
in scikit-learn-contrib/lightning. I would love to hear your thoughts on

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scikit-learn/attachments/20180222/285689e5/attachment.html>

More information about the scikit-learn mailing list