<div dir="ltr">Isn't Jérémie's project at <a href="https://github.com/jeremiedbb/scikit-learn_benchmarks">https://github.com/jeremiedbb/scikit-learn_benchmarks</a> meant to be doing this? What's its status? How does it relate to Tom's work?<div><br></div><div>(Can we please take <a href="http://scikit-learn.org/ml-benchmarks/">http://scikit-learn.org/ml-benchmarks/</a> offline?)</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Tue, 23 Jul 2019 at 00:17, Nicolas Hug <<a href="mailto:niourf@gmail.com">niourf@gmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF">
<p>I agree having benchmarks for non regression would be very
helpful. A seemingly simple change in Cython code can lead to
drastic performance drop.</p>
<p>I can't find it back but I think Jérémie has submitted an issue
about this?<br>
</p>
<div class="gmail-m_5876719045635966039moz-cite-prefix"><br>
</div>
<div class="gmail-m_5876719045635966039moz-cite-prefix">On 7/22/19 9:59 AM, Tom Augspurger
wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div>Thanks Adrin,</div>
<div dir="ltr"><br>
</div>
<div dir="ltr">A month or so ago I started running scikit-learn
benchmarks, but I had to disable them since they were taking
too long (longer than a day).
<div>I haven't had time to investigate why, but I assume it
was an issue with how I set them up.</div>
<div><br>
</div>
<div>Just FYI, I'm planning to include "maintain and improve
the benchmark running tools" as part of the pandas'
application for the CZI grant.</div>
<div>All that is in <a href="https://github.com/asv-runner" target="_blank">https://github.com/asv-runner</a>
(a mix of Ansible, Airflow, and GitHub bots). If anyone is
interested in (possibly) having funding</div>
<div>to work on this, feel free to reach out to me off list
and we can discuss things.</div>
<div><br>
</div>
<div>Tom</div>
</div>
</div>
<br>
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Mon, Jul 22, 2019 at 8:53
AM Adrin <<a href="mailto:adrin.jalali@gmail.com" target="_blank">adrin.jalali@gmail.com</a>> wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div class="gmail_default" style="font-size:small">Hi,</div>
<div class="gmail_default" style="font-size:small"><br>
</div>
<div class="gmail_default" style="font-size:small">There is
this [page](<a href="https://pandas.pydata.org/speed/scikit-learn/" target="_blank">https://pandas.pydata.org/speed/scikit-learn/</a>)
maintained by some of the pandas maintainers
(@TomAugspurger in particular), and it seems like a really
good idea to have an eye on the performance of different
benchmarks through time just in case a PR introduces some
major drawbacks.<br>
<br>
However, he doesn't have the bandwidth to maintain it much
more, and not really the hardware. I think it'd be a good
idea for us to have that, wanted to bring it up and see
what you think!</div>
<div class="gmail_default" style="font-size:small"><br>
</div>
<div class="gmail_default" style="font-size:small">Cheers,</div>
<div class="gmail_default" style="font-size:small">Adrin.<br>
</div>
</div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote>
</div>
<br>
<fieldset class="gmail-m_5876719045635966039mimeAttachmentHeader"></fieldset>
<pre class="gmail-m_5876719045635966039moz-quote-pre">_______________________________________________
scikit-learn mailing list
<a class="gmail-m_5876719045635966039moz-txt-link-abbreviated" href="mailto:scikit-learn@python.org" target="_blank">scikit-learn@python.org</a>
<a class="gmail-m_5876719045635966039moz-txt-link-freetext" href="https://mail.python.org/mailman/listinfo/scikit-learn" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a>
</pre>
</blockquote>
</div>
_______________________________________________<br>
scikit-learn mailing list<br>
<a href="mailto:scikit-learn@python.org" target="_blank">scikit-learn@python.org</a><br>
<a href="https://mail.python.org/mailman/listinfo/scikit-learn" rel="noreferrer" target="_blank">https://mail.python.org/mailman/listinfo/scikit-learn</a><br>
</blockquote></div>