![](https://secure.gravatar.com/avatar/ae8547ef993e0dd573bbe29bef562293.jpg?s=120&d=mm&r=g)
Hi, On Wed, Mar 1, 2023 at 1:15 PM David Menéndez Hurtado <davidmenhur@gmail.com> wrote:
This is the trajectory of each one plotted with the loss. It seems like Powell has a hard time navigating narrow valleys. BFGS, on the other hand, is locally fitting a parabola, which is a good fit for the loss function here.
This is what I get if I instead feed noiseless data, further illustrating the fact that Powell doesn't like ravines.
The code is here: https://gist.github.com/Dapid/1da960739b9e006f41a962607f6b1c54
I can't say why Scipy's fails and Octave doesn't, but hopefully this gives someone else an idea.
Thanks very much for doing all that - and posting the Gist. Exploring a little further - I tried all 5 Powell-type implementations in the PGFO package, script in: https://github.com/matthew-brett/powell-fails/blob/main/powell_pdfo.py All implemented routines get the correct answer, except 'cobyla', which fails with a suitable message: """ message: Return from cobyla because the objective function has been evaluated maxfev times. method: cobyla nfev: 1000 status: 3 success: False x: [2.1950324 0.48383331] """ And, interestingly enough, our own COBYLA method gets the wrong answer on the same problem, but believes it has succeeded: """ nav] In [12]: spo.minimize(calc_sse, start, args=(x, y), method='COBYLA') Out[12]: fun: 1.0762214436895654 maxcv: 0.0 message: 'Optimization terminated successfully.' nfev: 30 status: 1 success: True x: array([2.24641976, 0.46774596]) """ Cheers, Matthew