[Numpy-discussion] N dimensional dichotomy optimization

[Gael Varoquaux]

On Mon, Nov 22, 2010 at 11:12:26PM +0100, Matthieu Brucher wrote:
> It seems that a simplex is what you need.

Ha! I am learning new fancy words. Now I can start looking clever.

> > I realize that maybe I should rephrase my question to try and draw more
> > out of the common wealth of knowledge on this mailing list: what do
> > people suggest to tackle this problem? Guided by Matthieu's suggestion, I
> > have started looking at Powell's algorithm, and given the introduction of
> > www.damtp.cam.ac.uk/user/na/NA_papers/NA2007_03.pdf I am wondering
> > whether I should not investigate it. Can people provide any insights on
> > these problems.

> Indeed, Powell may also a solution. A simplex is just what is closer
> to what you hinted as an optimization algorithm.

I'll do a bit more reading.

> > PS: The reason I am looking at this optimization problem is that I got
> > tired of looking at grid searches optimize the cross-validation score on
> > my 3-parameter estimator (not in the scikit-learn, because it is way too
> > specific to my problems).

> Perhaps you may want to combine it with genetic algorithms. We also
> kind of combine grid search with simplex-based optimizer with
> simulated annealing in some of our global optimization problems, and I
> think I'll try at one point to introduce genetic algorithms instead of
> the grid search.

Well, in the scikit, in the long run (it will take a little while) I'd
like to expose other optimization methods then the GridSearchCV, so if
you have code or advice to give us, we'd certainly be interested.

Gael