Re: [SciPy-Dev] Consideration of differential evolution minimizer being added to scipy.optimize.
Hi, earlier in the discussion I had placed the DE code in https://github.com/andyfaff/DEsolver I have now cloned scipy, and made by additions at: https://github.com/andyfaff/scipy/blob/master/scipy/optimize/_differentialev... I have made PEP8 changes, and changed the random number generator changes, as suggested by Robert. I had a look at the basinhopping benchmarks at: https://github.com/js850/scipy/tree/benchmark_basinhopping/scipy/optimize/be... The main difference between basinhopping and Differential Evolution is that DE is population based. This makes it hard to determine when convergence has been reached. For example, I normally terminate when the standard deviation of the population energies, divided by the mean, is less than some tolerance. In comparison basinhopping terminates when the solution hasn't changed for a certain number of iterations. From some preliminary investigations (with those benchmarks) it seems that the DE algorithm finds the global minimum with a similar number of function evaluations, but then it takes many more function evaluations for the entire population to converge. For example, one outlier in the population could be enough to prevent termination. Minimization methods all have their pro's and con's. For me, DE is a good way of curvefitting scattering data (e.g. http://scripts.iucr.org/cgi-bin/paper?aj5178). In the case where function expense is not high, then DE is excellent. On the other hand, if it's extremely high, then another method may be appropriate. Perhaps an appropriate step would be to put a pull request in, so that comments on this topic can be archived on that PR? cheers, Andrew. -- _____________________________________ Dr. Andrew Nelson _____________________________________
Hi, earlier in the discussion I had placed the DE code in https://github.com/andyfaff/DEsolver
I have now cloned scipy, and made by additions at:
https://github.com/andyfaff/scipy/blob/master/scipy/optimize/_differentialev...
I have made PEP8 changes, and changed the random number generator changes, as suggested by Robert. I had a look at the basinhopping benchmarks at:
https://github.com/js850/scipy/tree/benchmark_basinhopping/scipy/optimize/be...
The main difference between basinhopping and Differential Evolution is that DE is population based. This makes it hard to determine when convergence has been reached. For example, I normally terminate when the standard deviation of the population energies, divided by the mean, is less than some tolerance. In comparison basinhopping terminates when the solution hasn't changed for a certain number of iterations. From some preliminary investigations (with those benchmarks) it seems that the DE algorithm finds the global minimum with a similar number of function evaluations, but then it takes many more function evaluations for the entire population to converge. For example, one outlier in the population could be enough to prevent termination. Determining convergence for a global optimizer is not trivial. For the general case the best you can do is say "I can't find a lower function value." The default behavior for basinhopping is to run for the specified number of iterations and make no claim about success or failure. There is an optional parameter `niter_success` which instructs the algorithm to stop if the lowest function value hasn't changed in a while. For benchmarking global optimization algorithms I
On 05/03/14 05:55, Andrew Nelson wrote: like to use functions where the global minimum is known and stop the optimizer when the function value is within some tolerance of that value.
Minimization methods all have their pro's and con's. For me, DE is a good way of curvefitting scattering data (e.g. http://scripts.iucr.org/cgi-bin/paper?aj5178). In the case where function expense is not high, then DE is excellent. On the other hand, if it's extremely high, then another method may be appropriate.
Perhaps an appropriate step would be to put a pull request in, so that comments on this topic can be archived on that PR?
cheers, Andrew.
participants (2)
-
Andrew Nelson -
Jacob Stevenson