[SciPy-User] fmin_bfgs stuck in infinite loop
josef.pktd at gmail.com
josef.pktd at gmail.com
Wed Oct 26 12:16:18 EDT 2011
On Wed, Oct 26, 2011 at 1:18 AM, Johann Cohen-Tanugi
<johann.cohen-tanugi at lupm.univ-montp2.fr> wrote:
> Hi Josef, can you provide me with the code with which you exercize the
> nan occurrence? It looks more reasonable than the brutal code the OP put
> in track, and could be used in optimize/tests/test_optimize.py
I just got the message.
bgfs is a variation on the original example
bgfs2 is for multiple parameters, current example has bound objective,
but one parameter goes to inf but bfgs produces "Optimization
terminated successfully." fmin hits maxiter or max function
evaluation
The scripts are a bit dirty, since I change around things a lot, and I
don't know which version triggered which bug.
If it's not clear, I can clean them up later.
I would like to get a collection of corner case problems, but the
umbrella maxiter limit will fix them all, I guess.
Thanks,
Josef
>
> best,
> Johann
>
> On 10/24/2011 10:14 PM, josef.pktd at gmail.com wrote:
>> On Mon, Oct 24, 2011 at 3:59 PM,<josef.pktd at gmail.com> wrote:
>>> tricky things these reply to all, forwarding to list
>>>
>>> ---------- Forwarded message ----------
>>> From:<josef.pktd at gmail.com>
>>> Date: Mon, Oct 24, 2011 at 3:52 PM
>>> Subject: Re: [SciPy-User] fmin_bfgs stuck in infinite loop
>>> To: johann.cohen-tanugi at lupm.in2p3.fr
>>>
>>>
>>> On Mon, Oct 24, 2011 at 3:43 PM, Johann Cohen-Tanugi
>>> <johann.cohen-tanugi at lupm.univ-montp2.fr> wrote:
>>>> indeed, see the email I just sent : for nan linesearch_wolfe1 does not exit
>>>> gracefully, so that Ralf's patch is never encountered.
>>> I'm not sure what's going on,
>>>
>>> I just copied the few lines from
>>> https://github.com/scipy/scipy/commit/a31acbf into my scipy 0.9 and
>>> the original example stops and I'm not able to produce an endless loop
>>> anymore when I try to change around with any of the examples, even
>>> when I start with a nan, it stops immediately. I only tried the one
>>> parameter example.
>> Nope, still endless in linesearch with -np.exp(-x/2.) -np.exp(-x**2)
>> the third iteration starts with nan as parameter and never returns
>> from the line search.
>>
>> So something like your original fix to exit a nan linesearch looks necessary.
>>
>> Josef
>>
>>
>>> Can you check that you actually run the code that has this fix?
>>>
>>> Josef
>>>
>>>
>>>> Johann
>>>>
>>>> On 10/24/2011 09:26 PM, josef.pktd at gmail.com wrote:
>>>>> On Mon, Oct 24, 2011 at 2:39 PM, Johann Cohen-Tanugi
>>>>> <johann.cohentanugi at gmail.com> wrote:
>>>>>> Dear Josef
>>>>>> On 10/24/2011 07:58 PM, josef.pktd at gmail.com wrote:
>>>>>>> On Mon, Oct 24, 2011 at 1:56 PM,<josef.pktd at gmail.com> wrote:
>>>>>>>> On Mon, Oct 24, 2011 at 1:50 PM, Johann Cohen-Tanugi
>>>>>>>> <johann.cohentanugi at gmail.com> wrote:
>>>>>>>>> Hello,
>>>>>>>>> the OP is a colleague of mine and I looked quickly at the code. The
>>>>>>>>> infinite
>>>>>>>>> loop in the OP's illustrating script comes from the "while 1" loop in
>>>>>>>>> l.144
>>>>>>>>> of linesearch.py : becuse phi0 is np.nan, phi1 is returned as np.nan
>>>>>>>>> as
>>>>>>>>> well, and the break condition is never met. There is an easy fix :
>>>>>>>>> while 1:
>>>>>>>>> stp, phi1, derphi1, task = minpack2.dcsrch(alpha1, phi1,
>>>>>>>>> derphi1,
>>>>>>>>> c1, c2, xtol, task,
>>>>>>>>> amin, amax, isave,
>>>>>>>>> dsave)
>>>>>>>>> if task[:2] == asbytes('FG') and not np.isnan(phi1):
>>>>>>>>> alpha1 = stp
>>>>>>>>> phi1 = phi(stp)
>>>>>>>>> derphi1 = derphi(stp)
>>>>>>>>> else:
>>>>>>>>> break
>>>>>>>>>
>>>>>>>>> but it is not a nice kludge.... Is there a better way to secure this
>>>>>>>>> while 1
>>>>>>>>> loop? I am sure I am not covering all possible pathological cases with
>>>>>>>>> adding "not np.isnan(phi1)" in the code above.
>>>>>>>> Is this still a problem with 0.10 ?
>>>>>>>> I thought this fixed it, https://github.com/scipy/scipy/commit/a31acbf
>>>>>> Well I am a complete newby with github, but I think I went to the head of
>>>>>> master before testing, and the problem is still there. I can see the code
>>>>>> snippet from https://github.com/scipy/scipy/commit/a31acbf in my local
>>>>>> copy,
>>>>>> and this is testing against +/-inf, not nan. Changing the OP's code to
>>>>>> test
>>>>>> against inf yields :
>>>>>> In [1]: run test_bgfs.py
>>>>>>
>>>>>> /home/cohen/sources/python/scipydev/lib/python2.6/site-packages/scipy/optimize/optimize.py:303:
>>>>>> RuntimeWarning: invalid value encountered in subtract
>>>>>> if (max(numpy.ravel(abs(sim[1:] - sim[0])))<= xtol \
>>>>>>
>>>>>> /home/cohen/sources/python/scipydev/lib/python2.6/site-packages/scipy/optimize/optimize.py:308:
>>>>>> RuntimeWarning: invalid value encountered in subtract
>>>>>> xr = (1 + rho)*xbar - rho*sim[-1]
>>>>>>
>>>>>> /home/cohen/sources/python/scipydev/lib/python2.6/site-packages/scipy/optimize/optimize.py:350:
>>>>>> RuntimeWarning: invalid value encountered in subtract
>>>>>> sim[j] = sim[0] + sigma*(sim[j] - sim[0])
>>>>>> fmin works [ inf]
>>>>>>
>>>>>> /home/cohen/sources/python/scipydev/lib/python2.6/site-packages/scipy/optimize/linesearch.py:132:
>>>>>> RuntimeWarning: invalid value encountered in subtract
>>>>>> alpha1 = min(1.0, 1.01*2*(phi0 - old_phi0)/derphi0)
>>>>>>
>>>>>> /home/cohen/sources/python/scipydev/lib/python2.6/site-packages/scipy/optimize/linesearch.py:308:
>>>>>> RuntimeWarning: invalid value encountered in subtract
>>>>>> alpha1 = min(1.0, 1.01*2*(phi0 - old_phi0)/derphi0)
>>>>>>
>>>>>> /home/cohen/sources/python/scipydev/lib/python2.6/site-packages/scipy/optimize/linesearch.py:417:
>>>>>> RuntimeWarning: invalid value encountered in subtract
>>>>>> B = (fb-D-C*db)/(db*db)
>>>>>> fmin_bfgs gets stuck in a loop [ nan]
>>>>>>
>>>>>> so it looks like your code solves the inf situation, but not the nan.
>>>>> It's not my fix (I'm still on scipy 0.9 and avoid bfgs because I don't
>>>>> want to have to kill my interpreter session)
>>>>>
>>>>> isfinite also checks for nans
>>>>>
>>>>>>>> np.isfinite(np.nan)
>>>>> False
>>>>>
>>>>> so there should be another reason that the linesearch doesn't return.
>>>>>
>>>>> Josef
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>>> Is http://projects.scipy.org/scipy/ticket/1542 the same?
>>>>>> yes it looks like a duplicate
>>>>>>> josef
>>>>>>>
>>>>>>>> Josef
>>>>>>>>
>>>>>>>>
>>>>>>>>> thoughts?
>>>>>>>>> Johann
>>>>>>>>>
>>>>>>>>> On 08/14/2011 01:38 AM, b9o2jnbm tsd71eam wrote:
>>>>>>>>>
>>>>>>>>> I have run into a frustrating problem where scipy.optimize.fmin_bfgs
>>>>>>>>> will
>>>>>>>>> get stuck in an infinite loop.
>>>>>>>>>
>>>>>>>>> I have submitted a bug report:
>>>>>>>>>
>>>>>>>>> http://projects.scipy.org/scipy/ticket/1494
>>>>>>>>>
>>>>>>>>> but would also like to see if anybody on this list has any suggestions
>>>>>>>>> or
>>>>>>>>> feedback.
>>>>>>>>>
>>>>>>>>> Thanks,
>>>>>>>>>
>>>>>>>>> --
>>>>>>>>> This message has been scanned for viruses and
>>>>>>>>> dangerous content by MailScanner, and is
>>>>>>>>> believed to be clean.
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> SciPy-User mailing list
>>>>>>>>> SciPy-User at scipy.org
>>>>>>>>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> SciPy-User mailing list
>>>>>>>>> SciPy-User at scipy.org
>>>>>>>>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>>>>>>>>
>>>>>>>>>
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-user
>>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: try_endless_bgfs2.py
Type: text/x-python
Size: 1454 bytes
Desc: not available
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20111026/fb24f45d/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: try_endless_bgfs.py
Type: text/x-python
Size: 1206 bytes
Desc: not available
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20111026/fb24f45d/attachment-0001.py>
More information about the SciPy-User
mailing list