[SciPy-User] (Possible) new optimization routines - scipy.optimize

[josef.pktd at gmail.com]

On Thu, Feb 14, 2013 at 4:01 PM, Andrea Gavana <andrea.gavana at gmail.com> wrote:
> Hi All,
>
>     as my team and I are constantly facing very hard/complex numerical
> optimization problems, I have taken a look at the various *global*
> optimization routines available in Python and I thought I could throw
> in a couple of algorithms I implemented, mostly drawing from my
> previous thesis work.
>
> I have implemented two routines (based on numpy and scipy), namely:
>
> - AMPGO: Adaptive Memory Programming for Global Optimization: this is my Python
>   implementation of the algorithm described here:
>
>   http://leeds-faculty.colorado.edu/glover/fred%20pubs/416%20-%20AMP%20(TS)%20for%20Constrained%20Global%20Opt%20w%20Lasdon%20et%20al%20.pdf
>
>   I have added a few improvements here and there based on my Master Thesis work
>   on the standard Tunnelling Algorithm of Levy, Montalvo and Gomez.

This could also be a good addition. similar to basinhopping.
>From my perspective, this kind of global optimizers are the most
promising, (compared to the evolutionary, ...)

>From a quick browse: Is the local optimizer fixed to a specific one,
or can it be any available solver as in basinhopping?

The only thing I might worry about that it only has 6 citations in
Google Scholar (which might not mean much if the optimizer is not
widely available).
Given that there seem to be many variations of this kind of
optimizers, it might be good to have some background on comparison
with similar optimizers.

If you have other comparisons of similar optimizers, it would be
useful to see them. Also given that you have a large benchmark suite,
you could compare it with the new basinhopping in scipy.optimize.


>
> - Firefly:  the Firefly algorithm, this is my Python implementation of
> the procedure
>   described here:
>
>   http://www.mathworks.com/matlabcentral/fileexchange/29693-firefly-algorithm

the fileexchange has a large number of "animal" optimizers, and I
doubt they are all good.

I think there needs to be a convincing case before adding any of them
should be added to scipy.
On the other hand, having them available outside of scipy would make
it easier to try them out.
(and see if fireflies, or bees or ants are doing better :)

thank you for proposing this, from a potential user.
I expect to try out basinhopping soon on estimation parameters of
mixture distributions.

Josef

>
>
> As it appears that most numerical optimization "experts" still report
> their benchmark results based on "CPU time" or "elapsed time" or
> similar meaningless performance indicators, I have built a fairly
> sizeable benchmark test suite to test various optimization routines.
> The test suite currently contains:
>
> - 18 one-dimensional test functions with multiple local/global minima;
> - 85 multivariate problems (where the number of independent variables
> ranges from 2 to 10), again with multiple local/global minima.
>
> The main page describing the rules, algorithms and motivation is here:
>
> http://infinity77.net/global_optimization/index.html
>
> Algorithms comparisons:
>
> http://infinity77.net/global_optimization/multidimensional.html
> http://infinity77.net/global_optimization/univariate.html
>
> Test functions:
>
> http://infinity77.net/global_optimization/test_functions.html
>
> The overall conclusion is that, while the Firefly method performances
> are average at best, AMPGO is superior to all the other algorithms I
> have tried. However, to be fair, it is an algorithm designed for
> low-dimensional optimization problems (i.e., 1-10 variables).
>
> I haven't published the code for the two algorithms (yet), as they are
> not quite up to the documentation standards of scipy. However, if
> there is interest from the community to add them (or one of them) to
> scipy.optimize I will be happy to polish them up and contribute to
> this great library. The same applies to the benchmark test suite.
>
>
> Andrea.
>
> "Imagination Is The Only Weapon In The War Against Reality."
> http://www.infinity77.net
>
> # ------------------------------------------------------------- #
> def ask_mailing_list_support(email):
>
>     if mention_platform_and_version() and include_sample_app():
>         send_message(email)
>     else:
>         install_malware()
>         erase_hard_drives()
> # ------------------------------------------------------------- #
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user