[Numpy-discussion] any interest in includinga second-ordergradient?

[Andrew Hawryluk]

> -----Original Message-----
> From: numpy-discussion-bounces at scipy.org [mailto:numpy-discussion-
> bounces at scipy.org] On Behalf Of David Warde-Farley
> Sent: 28 Oct 2008 10:15 PM
> To: Discussion of Numerical Python
> Subject: Re: [Numpy-discussion] any interest in includinga second-
> ordergradient?
> 
> On 28-Oct-08, at 5:57 PM, Fabrice Silva wrote:
> 
> > Are there some parts of the code that may be used only once to
> > calculate both the gradient and the second derivative (isn't it
> called
> > the hessian, at least in the N-d case) ?
> 
> Probably.  I'd imagine depends on your differencing scheme; central
> differences on the second derivative might imply a need to do two non-
> central differences to sample the gradient at +eps and -eps on each
> axis, respectively.

That is true, at least in theory. The formula for the first derivative
can be interpreted as the average between the slopes on either side of a
point while the second derivative is proportional to the difference
between those slopes. In practice, however, both formulas are simplified
algebraically such that you don't get a 'free' first derivative while
calculating the second. The forward and backward differences at the
edges are also quite different between the first and second order.

> And yes, the square matrix of second derivatives of an N-d scalar
> function is typically called the Hessian, but some of the properties
> one typically expects from a "Hessian" (i.e. symmetry) are not
> necessarily true unless all of the the partial second derivatives are
> continuous.
> 
> > If a common function would fasten the computation of the gradient
and
> > the hessian with a single call to a new function gradients(), it is
> > worth...
> > If the intent is just a reduction of the total length of the file
> > containing the gradient and gradient2 functions, I do not understand
> > why modifying the existent code. Why not creating a new function
> > hessian() having the same signature than gradient?
> 
> As Stefan mentioned, the number of functions that get included when
you
> type "from numpy import *" is already pretty large, and it'd be nice
> not to make it larger. Though you do raise a good point -- if it's
> known that the user wants both the gradient and the Hessian, and you
> can get the first for free while computing the second, then that's
> preferable to having to make two calls and duplicate work.
> 
> David
> _______________________________________________
> Numpy-discussion mailing list
> Numpy-discussion at scipy.org
> http://projects.scipy.org/mailman/listinfo/numpy-discussion

There could be a way to separate the first and second derivatives in a
way that lets the second take advantage of the work done by the first,
but it would yield only a small improvement on the second and it would
require a reduction in performance on the first. It would also make the
code much more confusing, and harder to extend in the future to higher
order derivatives.

On the whole it appears that Stefan's suggestion is the winner: one
function that returns one derivative of a specified order.

Andrew