[SciPy-User] optimize.minimize - help me understand arrays as variables (KURT PETERS)
KURT PETERS
peterskurt at msn.com
Mon Jan 19 20:41:36 EST 2015
> Date: Thu, 15 Jan 2015 08:00:01 -0700
> From: KURT PETERS <peterskurt at msn.com>
> Subject: Re: [SciPy-User] optimize.minimize - help me understand
> arrays as variables
> To: "scipy-user at scipy.org" <scipy-user at scipy.org>,
> "gdmcbain at freeshell.org" <gdmcbain at freeshell.org>
> Message-ID: <BLU172-W505EE00D73BEBCA063B4DED84E0 at phx.gbl>
> Content-Type: text/plain; charset="iso-8859-1"
>
> See below
> > Date: Thu, 15 Jan 2015 11:01:56 +1100
> > From: Geordie McBain <gdmcbain at freeshell.org>
> > Subject: Re: [SciPy-User] optimize.minimize - help me understand
> > arrays as variables (Andrew Nelson)
> > To: SciPy Users List <scipy-user at scipy.org>
> > Message-ID:
> > <CAB0gBH38Zx4Sp=zxxPVe_RjTqJ+ifHTQikW3WwLRYt+4o1Y1=Q at mail.gmail.com>
> > Content-Type: text/plain; charset=UTF-8
> >
> > 2015-01-13 10:26 GMT+11:00 KURT PETERS <peterskurt at msn.com>:
> > > > Date: Sun, 11 Jan 2015 17:55:32 -0700
> > >> From: KURT PETERS <peterskurt at msn.com>
> > >> Subject: [SciPy-User] optimize.minimize - help me understand arrays as
> > >> variables
> > >> To: "scipy-user at scipy.org" <scipy-user at scipy.org>
> > >> Message-ID: <BLU172-W37473350444550BD9CF90DD8430 at phx.gbl>
> > >> Content-Type: text/plain; charset="iso-8859-1"
> > >>
> > >> I'm trying to use scipy.optimize.minimize.
> > >> I've tried multiple "multivariate" methods that don't seem to actually
> > >> take multivariate data and derivatives. Can someone tell me how I can make
> > >> the multivariate part of the solver actually work?
> > >>
> > >> Here's an example:
> > >> My main function the following (typical length for N is 3):
> > >>
> > >> input guess is a x0=np.array([1,2,3])
> > >> the optimization function returns:
> > >> def calc_f3d(...):
> > >> f3d = np.ones((np.max([3,N]),1)
> > >> .... do some assignments to f3d[row,0] ....
> > >> return np.linalg.norm(f3d) # numpy.array that's 3x1
> > >>
> > >> The jacobian returns a Nx3 matrix:
> > >> def jacob3d(...):
> > >> df = np.ones((np.max([3,N]),3))
> > >> ... do some assignments to df[row,col]
> > >> return df # note numpy.array that's 3x3
> >
> > Hello. I think that this might be the problem here: the jac should
> > have the same shape as x, i.e. (N,), not (N, 3); the components of the
> > jac are the partial derivatives of fun with respect to the
> > corresponding components of x. Think of it as the gradient of the
> > objective.
> >
> > Here's a simple shape=(2,) example, taken from D. M. Greig's
> > Optimisation (1980, London: Longman). The exact minimum is 0 at [1,
> > 1].
> >
> > def f(x): # Greig (1980, p. 48)
> > return (x[1] - x[0]**2)**2 + (1 - x[0])**2
> >
> > def g(x): # ibid
> > return np.array([-4*x[0]*(x[1] - x[0]**2) - 2 + 2*x[0],
> > 2 * (x[1] - x[0]**2)])
> >
> > x = np.zeros(2)
> > print('Without Jacobian: ', minimize(f, x))
> > print('\nWith:', minimize(f, x, jac=g))
>
> Geordie,
> I don't think that's the case. Everything I've ever learned about the Jacobian is that it's the partials of each function with respect to each variable... so two equations with two unknowns, would yield a 2x2. Here's a wiki explaining what I mean:
> http://en.wikipedia.org/wiki/Jacobian_matrix_and_determinant
>
> If what you're saying is right, then the people that developed the function don't know what a Jacobian is. I would find that hard to believe.
>
> Kurt
>
>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: http://mail.scipy.org/pipermail/scipy-user/attachments/20150115/9a06ee97/attachment-0001.html
>
> ------------------------------
>
> Message: 3
> Date: Thu, 15 Jan 2015 16:45:26 +0100
> From: Da?id <davidmenhur at gmail.com>
> Subject: Re: [SciPy-User] optimize.minimize - help me understand
> arrays as variables
> To: SciPy Users List <scipy-user at scipy.org>
> Message-ID:
> <CAJhcF=3x+2Xgk+oa6jMJOuvETYPD446eQXMv2XyQh=P3j3ODKw at mail.gmail.com>
> Content-Type: text/plain; charset=UTF-8
>
> On 15 January 2015 at 16:00, KURT PETERS <peterskurt at msn.com> wrote:
> > Geordie,
> > I don't think that's the case. Everything I've ever learned about the
> > Jacobian is that it's the partials of each function with respect to each
> > variable... so two equations with two unknowns, would yield a 2x2.
>
> Ah, but for a minimisation problem you can only have one equation. You
> can't take the minimum of a vector function, because you can't say one
> vector is smaller than the other (or, in other words, there are many
> possible definitions of smaller, and you have to choose). |R^n is not
> ordered.
>
> In your case, your function to be minimised returns
> np.linalg.norm(f3d), that is, a single number, f(\vec x). Therefore,
> the jacobian is a vector: [d f(\vec x)/d x_0, d f(\vec x)/d x_1, d
> f(\vec x)/d x_2].
>
> I believe your results are incorrect because how you are defining f3d:
>
> f3d = np.ones((np.max([3,N]),1)
>
> There is no need for the extra dimension, just
>
> f3d = np.ones(max(3, N))
>
> And do the assignments to f3d[row]
>
>
> /David.
>
>
> ------------------------------
>
> Message: 4
> Date: Thu, 15 Jan 2015 10:50:27 -0700
> From: KURT PETERS <peterskurt at msn.com>
> Subject: [SciPy-User] 2. Re: optimize.minimize - help me understand
> arrays as variables
> To: "scipy-user at scipy.org" <scipy-user at scipy.org>,
> "gdmcbain at freeshell.org" <gdmcbain at freeshell.org>
> Message-ID: <BLU172-W3347A0315DC8043344F9C8D84E0 at phx.gbl>
> Content-Type: text/plain; charset="iso-8859-1"
>
> > Date: Thu, 15 Jan 2015 11:01:56 +1100
> > From: Geordie McBain <gdmcbain at freeshell.org>
> > Subject: Re: [SciPy-User] optimize.minimize - help me understand
> > arrays as variables (Andrew Nelson)
> > To: SciPy Users List <scipy-user at scipy.org>
> > Message-ID:
> > <CAB0gBH38Zx4Sp=zxxPVe_RjTqJ+ifHTQikW3WwLRYt+4o1Y1=Q at mail.gmail.com>
> > Content-Type: text/plain; charset=UTF-8
> >
> > 2015-01-13 10:26 GMT+11:00 KURT PETERS <peterskurt at msn.com>:
> > > > Date: Sun, 11 Jan 2015 17:55:32 -0700
> > >> From: KURT PETERS <peterskurt at msn.com>
> > >> Subject: [SciPy-User] optimize.minimize - help me understand arrays as
> > >> variables
> > >> To: "scipy-user at scipy.org" <scipy-user at scipy.org>
> > >> Message-ID: <BLU172-W37473350444550BD9CF90DD8430 at phx.gbl>
> > >> Content-Type: text/plain; charset="iso-8859-1"
> > >>
> > >> I'm trying to use scipy.optimize.minimize.
> > >> I've tried multiple "multivariate" methods that don't seem to actually
> > >> take multivariate data and derivatives. Can someone tell me how I can make
> > >> the multivariate part of the solver actually work?
> > >>
> > >> Here's an example:
> > >> My main function the following (typical length for N is 3):
> > >>
> > >> input guess is a x0=np.array([1,2,3])
> > >> the optimization function returns:
> > >> def calc_f3d(...):
> > >> f3d = np.ones((np.max([3,N]),1)
> > >> .... do some assignments to f3d[row,0] ....
> > >> return np.linalg.norm(f3d) # numpy.array that's 3x1
> > >>
> > >> The jacobian returns a Nx3 matrix:
> > >> def jacob3d(...):
> > >> df = np.ones((np.max([3,N]),3))
> > >> ... do some assignments to df[row,col]
> > >> return df # note numpy.array that's 3x3
> >
> > Hello. I think that this might be the problem here: the jac should
> > have the same shape as x, i.e. (N,), not (N, 3); the components of the
> > jac are the partial derivatives of fun with respect to the
> > corresponding components of x. Think of it as the gradient of the
> > objective.
> >
> > Here's a simple shape=(2,) example, taken from D. M. Greig's
> > Optimisation (1980, London: Longman). The exact minimum is 0 at [1,
> > 1].
> >
> > def f(x): # Greig (1980, p. 48)
> > return (x[1] - x[0]**2)**2 + (1 - x[0])**2
> >
> > def g(x): # ibid
> > return np.array([-4*x[0]*(x[1] - x[0]**2) - 2 + 2*x[0],
> > 2 * (x[1] - x[0]**2)])
> >
> > x = np.zeros(2)
> > print('Without Jacobian: ', minimize(f, x))
> > print('\nWith:', minimize(f, x, jac=g))
>
> I'm going to try the root function. I just saw the words "multivariate scalar function" in the documentation for minimize. Maybe my assumption was that it could handle multiple functions. As I read further, there's a distinction to multiple functions in the "Root" function, such as with Krylov. I'm going to see if that behaves the way I would expect. Perhaps, I misunderstood the documentation.
> I'm going to try that and let the group know how that worked.
> Kurt
>
I tried to use the root function.
I think there's a problem in the scipy/optimize/minpack.py.
In def _check_func, the function does a "len(output_shape)"
on line 20. If one returns a proper (3,1) instead of (3,), the "len" will return a "1." This gives the function the false impression that the size is wrong. First off, those checks conducted seem to have a variety of "shoe-horned" checks:
minpack.py:def _check_func(checker, argname, thefunc, x0, args, numinputs, output_shape=None):
minpack.py: shape, dtype = _check_func('fsolve', 'func', func, x0, args, n, (n,))
minpack.py: _check_func('fsolve', 'fprime', Dfun, x0, args, n, (n,n))
minpack.py: shape, dtype = _check_func('leastsq', 'func', func, x0, args, n)
minpack.py: _check_func('leastsq', 'Dfun', Dfun, x0, args, n, (n,m))
minpack.py: _check_func('leastsq', 'Dfun', Dfun, x0, args, n, (m,n))
Why is fsolve and _root_hybr forcing "output_shape" into (n,1)? Why don't they actually use the shape of the array passed? Why are they forcing the input to be flattened instead of accounting for both (3,1) and (3,) possibilities? Doesn't seem logical.
Kurt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.scipy.org/pipermail/scipy-user/attachments/20150119/061efcdb/attachment.html>
More information about the SciPy-User
mailing list