<div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Sep 7, 2016 at 6:23 AM, Sylvain Corlay <span dir="ltr"><<a href="mailto:sylvain.corlay@gmail.com" target="_blank">sylvain.corlay@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>I understand (especially about the scikits), although I was surprised by some recently added features in scipy, such as differential evolution</div><div><div><br></div><div>DE is more of a "recipe" which has been studied empirically for which there is no theoretical convergence rate. Even though it may "work well" for certain problems, it causes some issues such as defining on which basis it should even be "improved", and what should be the foundation for a change. (Recently a result on convergence in probability (with no error rate) on a bounded domain has been published, but no result exists on the speed of convergence afaik...) <br></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><br></div><div>Evolutionary optimization is definitely cool and may work strikingly well for certain problems, but I was surprised that it got elected to inclusion in scipy. DE would have been a nice seed for a scikit-evolution.</div><div><br></div><div>For those interested in stochastic algorithms for global multivariate optimization for which we have proven convergence results and an abundant literature, we have at least the two following categories of methods</div><div><br></div><div> - <b>MCMC</b> (Markov Chain Monte Carlo) which comprises simulated annealing, for which there are nice results of convergence in distribution.</div><div> - <b>Robbins-Monro</b> methods, which comprise stochastic gradient methods, for which we have almost-sure convergence and central-limit - type theorems.</div></div></div></blockquote><div><br>I think it's fair to summarize the reason for the current status and inclusion of DE as: theoretical convergence rates and papers are nice, but code that actually works on a good set of benchmarks is way more important. <br><br>Some years ago we had only bad global optimizers. We did have simulated annealing, but it just didn't work for most problems. No one stepped up to improve it, so we deprecated and removed it.<br><br>DE was benchmarked quite thoroughly (on the benchmark functions from benchmarks/benchmarks/test_go_benchmark_functions.py) and came out looking good. It solved problems that the only other good optimizer we had (basinhopping) did not do well on. So that means there was value in adding it.<br> <br></div><div>Cheers,<br></div><div>Ralf<br><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div><br></div></div><div>This sort of raises the question of the scope of scipy. Should scipy go through the same sort of "big split" as Jupyter or more à la d3js? Is amount of specialized knowledge required to understand a methods part of what defines the line of division in what should be or not be in scipy?</div><span class="gmail-HOEnZb"><font color="#888888"><div><br></div><div>Sylvain</div></font></span><div><div class="gmail-h5"><div><br></div><div class="gmail_extra"><div class="gmail_quote">On Tue, Sep 6, 2016 at 8:08 PM, Jacob Vanderplas <span dir="ltr"><<a href="mailto:jakevdp@cs.washington.edu" target="_blank">jakevdp@cs.washington.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">As for adding more functionality along those lines, I would advocate creation of a new package or perhaps a scikit. As we've seen with scikit-learn, useful features can filter-up into scipy (e.g. sparse.csgraph) and the development of new features within an independent package is *much* easier than development from within a scipy PR.<div> Jake</div></div><div class="gmail_extra"><span><br clear="all"><div><div><div dir="ltr"><div><div dir="ltr"> <font size="1">Jake VanderPlas</font></div><div dir="ltr"><font size="1"> Senior Data Science Fellow<br></font><div><div><font size="1"> Director of Research in Physical Sciences</font></div><div><font size="1"> </font><span style="font-size:x-small">University of Washington </span><span style="font-size:x-small">eScience Institute</span></div></div></div></div></div></div></div>
<br></span><div><div><div class="gmail_quote">On Tue, Sep 6, 2016 at 11:03 AM, Sylvain Corlay <span dir="ltr"><<a href="mailto:sylvain.corlay@gmail.com" target="_blank">sylvain.corlay@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">Would you guys consider in scope for scipy to have implementation of faster nearest neighbor search methods than KdTree? <div><br></div><div>Some methods are fairly simple... e.g principal axis tree which use the principal direction of the dataset to split the dataset into smaller subsets. As soon as intrinsic dimensionality is significantly smaller than the dimension of the space, it is significantly faster. </div><div><br></div><div>Besides, only having to compute the (an approximate) principal axis is much faster than doing an actual PCA.<br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Sep 6, 2016 at 4:14 AM, Jacob Vanderplas <span dir="ltr"><<a href="mailto:jakevdp@cs.washington.edu" target="_blank">jakevdp@cs.washington.edu</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr">From my own casual benchmarks, the new scipy cKDTree is much faster than any of the scikit-learn options, though it still only supports axis-aligned euclidean-like metrics (where sklearn's BallTree supports dozens of additional metrics). The cKDTree also has a limited range of query types compared to scikit-learn's trees,<div> Jake</div></div><div class="gmail_extra"><br clear="all"><div><div><div dir="ltr"><div><div dir="ltr"> <font size="1">Jake VanderPlas</font></div><div dir="ltr"><font size="1"> Senior Data Science Fellow<br></font><div><div><font size="1"> Director of Research in Physical Sciences</font></div><div><font size="1"> </font><span style="font-size:x-small">University of Washington </span><span style="font-size:x-small">eScience Institute</span></div></div></div></div></div></div></div><div><div>
<br><div class="gmail_quote">On Mon, Sep 5, 2016 at 12:46 AM, Daπid <span dir="ltr"><<a href="mailto:davidmenhur@gmail.com" target="_blank">davidmenhur@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><span>On 4 September 2016 at 23:00, Robert Lucente <<a href="mailto:rlucente@pipeline.com" target="_blank">rlucente@pipeline.com</a>> wrote:<br>
> Please note that I am a newbie and just a lurker.<br>
><br>
> I noticed in a recent email that cKDTree was mentioned.<br>
><br>
> Q: What is the relationship if any between SciPy an scikit-learn when it comes to cKDTree?<br>
><br>
> The reason that I ask are the following 2 links<br>
><br>
> <a href="https://jakevdp.github.io/blog/2013/04/29/benchmarking-nearest-neighbor-searches-in-python/" rel="noreferrer" target="_blank">https://jakevdp.github.io/blog<wbr>/2013/04/29/benchmarking-neare<wbr>st-neighbor-searches-in-python<wbr>/</a><br>
><br>
> <a href="https://github.com/scikit-learn/scikit-learn/issues/3682" rel="noreferrer" target="_blank">https://github.com/scikit-lear<wbr>n/scikit-learn/issues/3682</a><br>
<br>
</span>Note that these benchmarks are from 2013 and 2014. Scipy's KDTree has<br>
seen its performance recently improved, twice. Scikit's last update to<br>
its KDTree was in 2015. So, we need to run the benchmarks again.<br>
<br>
/David.<br>
<div><div>______________________________<wbr>_________________<br>
SciPy-Dev mailing list<br>
<a href="mailto:SciPy-Dev@scipy.org" target="_blank">SciPy-Dev@scipy.org</a><br>
<a href="https://mail.scipy.org/mailman/listinfo/scipy-dev" rel="noreferrer" target="_blank">https://mail.scipy.org/mailman<wbr>/listinfo/scipy-dev</a><br>
</div></div></blockquote></div><br></div></div></div>
<br>______________________________<wbr>_________________<br>
SciPy-Dev mailing list<br>
<a href="mailto:SciPy-Dev@scipy.org" target="_blank">SciPy-Dev@scipy.org</a><br>
<a href="https://mail.scipy.org/mailman/listinfo/scipy-dev" rel="noreferrer" target="_blank">https://mail.scipy.org/mailman<wbr>/listinfo/scipy-dev</a><br>
<br></blockquote></div><br></div></div></div></div>
<br>______________________________<wbr>_________________<br>
SciPy-Dev mailing list<br>
<a href="mailto:SciPy-Dev@scipy.org" target="_blank">SciPy-Dev@scipy.org</a><br>
<a href="https://mail.scipy.org/mailman/listinfo/scipy-dev" rel="noreferrer" target="_blank">https://mail.scipy.org/mailman<wbr>/listinfo/scipy-dev</a><br>
<br></blockquote></div><br></div></div></div>
<br>______________________________<wbr>_________________<br>
SciPy-Dev mailing list<br>
<a href="mailto:SciPy-Dev@scipy.org" target="_blank">SciPy-Dev@scipy.org</a><br>
<a href="https://mail.scipy.org/mailman/listinfo/scipy-dev" rel="noreferrer" target="_blank">https://mail.scipy.org/mailman<wbr>/listinfo/scipy-dev</a><br>
<br></blockquote></div><br></div></div></div></div>
<br>______________________________<wbr>_________________<br>
SciPy-Dev mailing list<br>
<a href="mailto:SciPy-Dev@scipy.org">SciPy-Dev@scipy.org</a><br>
<a href="https://mail.scipy.org/mailman/listinfo/scipy-dev" rel="noreferrer" target="_blank">https://mail.scipy.org/<wbr>mailman/listinfo/scipy-dev</a><br>
<br></blockquote></div><br></div></div>