Hi, I have a few questions about scipy.optimize.anneal. I'm using 0.9.0 under Debian Squeeze with Python 2.6.6, but the documentation for anneal is the same as for 0.10.0. Firstly, I'm confused about the return values — the docs say that the return values are: xmin (ndarray), retval (int), Jmin (float), T (float), feval (int), iters (int), accept (int) ...however, running the attached script gives (array([-10.16682436, 6.46000538]), 12.045579564382059, 0.72724961763274643, 851, 16, 180, 5) [<type 'numpy.ndarray'>, <type 'numpy.float64'>, <type 'numpy.float64'>, <type 'int'>, <type 'int'>, <type 'int'>, <type 'int'>] Note that the types don't match and there's no value that could correspond to the documented options for retval! Am I reading things wrong, or are these inconsistent with the docs? Secondly, since annealing is a random process, is there any way to control the source of random numbers it uses? Should I seed some global RNG, and if so, which one? Thirdly, note that the script actually fails to find the optimal parameters, despite being given them as the starting point. I know that annealing is a far from reliable process, but this behaviour still seems a little strange given the simplicity of the function I'm using here. Are there additional constraints I should be supplying, or some other way to guide the annealing process? Fourthly, in my actual application I have a function to optimise that is invariant to the overall scale of the inputs, ie. f(a*array([x,y,z])) is the same for all non-zero values of "a". Is there a risk of anneal failing because of this degeneracy? Is there a way to structure the problem to avoid this? (Note that any component could legitimately be zero, so I can't just fix x = 1 ... I think). Cheers, Jason
On Tue, Jul 26, 2011 at 5:31 AM, Jason Heeris <jason.heeris@gmail.com>wrote:
Hi,
I have a few questions about scipy.optimize.anneal. I'm using 0.9.0 under Debian Squeeze with Python 2.6.6, but the documentation for anneal is the same as for 0.10.0.
Firstly, I'm confused about the return values — the docs say that the return values are: xmin (ndarray), retval (int), Jmin (float), T (float), feval (int), iters (int), accept (int)
...however, running the attached script gives
(array([-10.16682436, 6.46000538]), 12.045579564382059, 0.72724961763274643, 851, 16, 180, 5)
[<type 'numpy.ndarray'>, <type 'numpy.float64'>, <type 'numpy.float64'>, <type 'int'>, <type 'int'>, <type 'int'>, <type 'int'>]
Note that the types don't match and there's no value that could correspond to the documented options for retval! Am I reading things wrong, or are these inconsistent with the docs?
The docs are wrong. Judging from the source code, it appears that retval is the final entry in the tuple of returned values and the rest are just shifted up one. The floats in the docs are, apparently, really np.floats.
Secondly, since annealing is a random process, is there any way to control the source of random numbers it uses? Should I seed some global RNG, and if so, which one?
It appears from the source code that all sources of randomness in the function come from calls to random number generators in numpy.random. So numpy.random.seed should be the only seed you need to set. (And this seemed to be the case after testing it on your sample function.)
Thirdly, note that the script actually fails to find the optimal parameters, despite being given them as the starting point. I know that annealing is a far from reliable process, but this behaviour still seems a little strange given the simplicity of the function I'm using here. Are there additional constraints I should be supplying, or some other way to guide the annealing process?
The default schedule is 'fast', and it does seem to give really lousy results. It seems to cool too quickly. If you try the boltzmann or cauchy schedules, they take longer but give much better estimates of the minimum.
Fourthly, in my actual application I have a function to optimise that is invariant to the overall scale of the inputs, ie. f(a*array([x,y,z])) is the same for all non-zero values of "a". Is there a risk of anneal failing because of this degeneracy? Is there a way to structure the problem to avoid this? (Note that any component could legitimately be zero, so I can't just fix x = 1 ... I think).
Assuming your function is continuous, I think the biggest problem might be that your function doesn't increase as x becomes big. So your samples could wander to infinity. I'd be curious what would happen if you simply scaled each new sample so it had unit length. But to do that you'll need to download scipy's source and make the modifications yourself. Another possibility is simply giving reasonable upper and lower bounds for x using lower and upper. -Chris Jordan-Squire
Cheers, Jason
_______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
On 26 July 2011 23:46, Christopher Jordan-Squire <cjordan1@uw.edu> wrote:
The docs are wrong. Judging from the source code, it appears that retval is the final entry in the tuple of returned values and the rest are just shifted up one. The floats in the docs are, apparently, really np.floats.
Thanks for all the info — there's only one detail I'm still puzzled about: what would a "retval" of 5 indicate? — Jason
On Tue, Jul 26, 2011 at 11:18 AM, Jason Heeris <jason.heeris@gmail.com>wrote:
On 26 July 2011 23:46, Christopher Jordan-Squire <cjordan1@uw.edu> wrote:
The docs are wrong. Judging from the source code, it appears that retval is the final entry in the tuple of returned values and the rest are just shifted up one. The floats in the docs are, apparently, really np.floats.
Thanks for all the info — there's only one detail I'm still puzzled about: what would a "retval" of 5 indicate?
As advertised by the warning, it just seems to indicate that the point the annealing cooled to wasn't the lowest point it encountered. Just a warning, I think, that things are pretty screwed up and you should try a different temperature schedule. But it's strange that's not in the docs. -Chris Jordan-Squire
— Jason _______________________________________________ SciPy-User mailing list SciPy-User@scipy.org http://mail.scipy.org/mailman/listinfo/scipy-user
On 26 July 2011 18:31, Jason Heeris <jason.heeris@gmail.com> wrote:
Fourthly, in my actual application I have a function to optimise that is invariant to the overall scale of the inputs, ie. f(a*array([x,y,z])) is the same for all non-zero values of "a". Is there a risk of anneal failing because of this degeneracy? Is there a way to structure the problem to avoid this? (Note that any component could legitimately be zero, so I can't just fix x = 1 ... I think).
Just in case anyone else is interested, I realised the answer to this after I'd had my morning coffee. Since the origin is not a valid input, I really just want to anneal over the points on the surface of a 5-sphere of arbitrary non-zero radius. So I can fix the radius to be 1 and constrain my input vectors to have five coordinates which are angles, ie. hyperspherical coordinates (ala. http://en.wikipedia.org/wiki/N-sphere#Hyperspherical_coordinates) — Jason
participants (2)
-
Christopher Jordan-Squire
-
Jason Heeris