[scikit-learn] L-BFGS in MLPClassifier
mathieu at mblondel.org
Fri Aug 25 21:33:06 EDT 2017
Thanks for this email. It is always nice to hear about success stories.
I assume the guilty party is Issam Laradji, as you can see from his Google
Summer of Code blog post:
L-BFGS is indeed usually a good default choice for medium-scale datasets.
It doesn't require any step size tuning and I found recently that it works
well for poorly conditioned problems.
You can also see a blog post by Nicolas Le Roux praising L-BFGS here:
On Sat, Aug 26, 2017 at 12:40 AM, Dr. Mario Michael Krell <
mario.michael.krell at gmail.com> wrote:
> To whoever programmed the MLPClassifier (with the L-BFGS solver),
> I just wanted to personally thank you and if I get your name(s), I would
> mention it/them in my paper additionally to the mandatory sklearn citation.
> I hope that sklearn will be keeping this algorithm forever in their
> library despite the increasing amount of established deep learning
> libraries that seem to make this code obsolete. For my small scale, more
> theoretic analysis, it worked much better than any other algorithm and I
> would not have gotten such surprising results. Due to the high quality
> implementation, the integration of a much better solver than SGD, and the
> respective good documentation, I could show empirically how the VC
> dimension and another property of MLPs (MacKay dimension) actually scale
> linear with the number of edges in the respective graph which helped us to
> provide a new much more strict upper bound (https://arxiv.org/abs/1708.
> 06019). This would have not been possible with other implementations. If
> there is an interest by the developers, I could try to contribute a
> tutorial documentation for sklearn. Just let me know.
> Thank you a lot!!!
> scikit-learn mailing list
> scikit-learn at python.org
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the scikit-learn