<div dir="ltr">Hey Gurhan,<div><br></div><div>sklearn doesn't really neatly separate optimizers from the models they optimize at the level of API (except in a few cases). In order to make the package more friendly to newer user, each model has excellent optimizer defaults that you can use, and only in a few cases does it make sense to tweak the optimization routines (for example, SAGA if you have a very large dataset when doing logistic regression).</div><div><br></div><div>There is a fantastic library called lightning where the optimization routines are first class citizens: <a href="http://contrib.scikit-learn.org/lightning/" target="_blank">http://contrib.scikit-learn.org/lightning/</a> - you can take a look there. However, lightning focuses on convex optimization, so most algorithms have provable convergence rates.</div><div><br></div><div>Good luck!</div><br><div class="gmail_quote"><div dir="ltr">On Mon, 10 Jul 2017 at 09:05 Jacob Schreiber <<a href="mailto:jmschreiber91@gmail.com" target="_blank">jmschreiber91@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Howdy<div><br></div><div>This question and the one right after in the FAQ are probably relevant re: inclusion of new algorithms: <a href="http://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms" target="_blank">http://scikit-learn.org/stable/faq.html#what-are-the-inclusion-criteria-for-new-algorithms</a>. The gist is that we only include well established algorithms, and there are no end to those. I think it is unlikely that a PR will get merged with a cutting edge new algorithm, as the scope of scikit-learn isn't necessary "the latest" as opposed to "the classics." You may also consider writing a scikit-contrib package that basically creates what you're interested in in scikit-learn format, but external to the project. We'd be more than happy to link to it. If the algorithm becomes a smashing success over time, we'd reconsider adding it to the main code base.</div><div><br></div><div>As to your first question, you should check out how the current optimizers are written for the algorithm you're interested in. I don't think there's a plug and play way to drop in your own optimizer like many deep learning packages support, unfortunately. You'd probably have to modify the code directly to support your own.</div><div><br></div><div>Let me know if you have any other questions.</div></div><div dir="ltr"><div><br></div><div>Jacob</div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Jul 10, 2017 at 7:58 AM, Gürhan Ceylan <span dir="ltr"><<a href="mailto:grhanceylan@gmail.com" target="_blank">grhanceylan@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><font face="arial, helvetica, sans-serif">Hi everyone,</font><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif"><span style="color:rgb(36,39,41)">I am wondering, How can I use external optimization algorithms with </span>scikit-learn, for instance <a href="http://scikit-learn.org/stable/modules/neural_networks_supervised.html#algorithms" target="_blank">neural network</a>,<font color="#242729"> instead of defined algorithms ( Stochastic Gradient Descent, Adam, or L-BFGS). </font><br></font></div><div><span style="color:rgb(36,39,41)"><font face="arial, helvetica, sans-serif"><br></font></span></div><div><font color="#242729" face="arial, helvetica, sans-serif">Furthermore, I want to introduce a new unconstrained optimization algorithm to scikit-learn, implementation of the algorithm and related</font><font face="arial, helvetica, sans-serif"><span style="color:rgb(36,39,41)"> paper can be found </span><a href="https://github.com/sibirbil/PMBSolve" target="_blank">here</a>. </font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">I couldn't find any <a href="http://scikit-learn.org/stable/developers/contributing.html" target="_blank">explanation</a>, about the situation. Do you have defined procedure to make such kind of contributions? If this is not the case, How should I start to make such a proposal/contribution ?</font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">Kind regards,</font></div><div><font face="arial, helvetica, sans-serif"><br></font></div><div><font face="arial, helvetica, sans-serif">Gürhan C.</font></div><div><font face="arial, helvetica, sans-serif"><br></font></div></div>
<br>_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
<br></blockquote></div><br></div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote></div></div>