Re: [SciPy-Dev] Consideration of differential evolution minimizer being added to scipy.optimize.
Dear all, I've put some more work into the differential evolution (another global optimizer) code I would like to contribute to scipy.optimize. Since this is the first time I'm contributing to scipy I'd would like a code review, with a view to putting in a pull request in the near future. The changes/additions I've made are at (under files changed): https://github.com/andyfaff/scipy/compare/scipy:master...master Some of the comments when I first discussed this were for benchmarking DE to see if it had comparable performance to basinhopping. To that end I've added some code and extra test functions (http://en.wikipedia.org/wiki/Test_functions_for_optimization) for benchmarking global optimizers. You can see the output of the benchmarking at: https://gist.github.com/andyfaff/9439229
From the test functions differential_evolution doesn't do too badly, especially when no gradient is available for the function to be minimized. Having said that, I do recognise that you sometimes need to tune the optimizer for a specific problem.
My specific motivation for adding differential_evolution is for curve_fitting. For most of my problems function expense isn't enormous, so I don't regard number of function evaluations to be all that important. In those cases finding the global minimum is the key. I've tried comparing basinhopping to DE for harder curvefitting problems and in the example I tried the number of function evaluations was a lot smaller for DE (http://www.itl.nist.gov/div898/strd/index.html - Thurber). (One can always use Levenberg Marquardt to 'polish' the end result) Of course, there are problems where function expense is enormous. However, if that is truly horrendous one can always use the callback function to terminate if you think one is getting towards a solution. cheers, Andrew. -- _____________________________________ Dr. Andrew Nelson _____________________________________
On Sat, Mar 8, 2014 at 10:55 PM, Andrew Nelson <andyfaff@gmail.com> wrote:
Dear all, I've put some more work into the differential evolution (another global optimizer) code I would like to contribute to scipy.optimize. Since this is the first time I'm contributing to scipy I'd would like a code review, with a view to putting in a pull request in the near future.
The changes/additions I've made are at (under files changed): https://github.com/andyfaff/scipy/compare/scipy:master...master
Could you send a PR? That's easier to comment on, and will be preserved better for future reference. Question that the docstring doesn't answer: - why are there so many options for `DEstrategy` (and what's the default)? - `mutation` and `regeneration` aren't explained, do I need to tweak those as a user? Some minor comments: - please reference the original paper (format as in https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt) instead of linking to someone's homepage. - there's ``**options`` in the signature, but it's not explained and doesn't appear to be needed. - tests: don't use plain assert, use np.testing.assert_ instead - tests: don't import unittest, only numpy.testing - tests: test cases shouldn't have docstrings because that messes up nose output, replace with comments
Some of the comments when I first discussed this were for benchmarking DE to see if it had comparable performance to basinhopping. To that end I've added some code and extra test functions (http://en.wikipedia.org/wiki/Test_functions_for_optimization) for benchmarking global optimizers. You can see the output of the benchmarking at: https://gist.github.com/andyfaff/9439229
So it looks like on average DE has worse performance, but there are problems where it does significantly better than basinhopping. That's also the conclusion one could draw from http://infinity77.net/global_optimization/multidimensional.html. Which may be an OK motivation to add it. Cheers, Ralf
From the test functions differential_evolution doesn't do too badly, especially when no gradient is available for the function to be minimized. Having said that, I do recognise that you sometimes need to tune the optimizer for a specific problem.
My specific motivation for adding differential_evolution is for curve_fitting. For most of my problems function expense isn't enormous, so I don't regard number of function evaluations to be all that important. In those cases finding the global minimum is the key. I've tried comparing basinhopping to DE for harder curvefitting problems and in the example I tried the number of function evaluations was a lot smaller for DE (http://www.itl.nist.gov/div898/strd/index.html - Thurber). (One can always use Levenberg Marquardt to 'polish' the end result)
Of course, there are problems where function expense is enormous. However, if that is truly horrendous one can always use the callback function to terminate if you think one is getting towards a solution.
cheers, Andrew.
-- _____________________________________ Dr. Andrew Nelson
_____________________________________ _______________________________________________ SciPy-Dev mailing list SciPy-Dev@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-dev
Hi Andrew, There is another Python implementation for Differential Evolution called Sherpa, which has been used in X-Ray Astronomy. See http://cxc.cfa.harvard.edu/sherpa/ Cheers, Clark
participants (3)
-
Andrew Nelson -
Ralf Gommers -
Zhang Jiang