<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:"Arial Black";
panose-1:2 11 10 4 2 1 2 2 2 4;}
@font-face
{font-family:Georgia;
panose-1:2 4 5 2 5 4 5 2 3 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
tt
{mso-style-priority:99;
font-family:"Courier New";}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.apple-converted-space
{mso-style-name:apple-converted-space;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:588275218;
mso-list-template-ids:1645786752;}
@list l1
{mso-list-id:1366440852;
mso-list-template-ids:-688893398;}
@list l2
{mso-list-id:1603488348;
mso-list-template-ids:26777226;}
@list l3
{mso-list-id:1618289856;
mso-list-template-ids:121826558;}
@list l4
{mso-list-id:1717462733;
mso-list-template-ids:374132554;}
@list l5
{mso-list-id:1963219998;
mso-list-template-ids:-2023848202;}
@list l5:level1
{mso-level-start-at:2;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="color:black">For your information I have reimplemented the approx._fprime and approx._hess code found in statsmodels and added the epsilon extrapolation<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">method of Wynn. The result you can see here:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><a href="https://github.com/pbrod/numdifftools/blob/master/numdifftools/nd_cstep.py"><span style="color:black">https://github.com/pbrod/numdifftools/blob/master/numdifftools/nd_cstep.py</span></a><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">I have also compared the accuracy and runtimes for the different alternatives here:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><a href="https://github.com/pbrod/numdifftools/blob/master/numdifftools/run_benchmark.py"><span style="color:black">https://github.com/pbrod/numdifftools/blob/master/numdifftools/run_benchmark.py</span></a></span><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">Personally I like the class interface better than the functional one because you can pass the resulting object as function to other methods/functions and these functions/methods do not need to know what it does
behind the scenes or what options are used. This simple use case is exemplified here:
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">>>> g = lambda x: 1./x<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">>>> dg = Derivative(g, **options)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">>>> my_plot(dg)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">>>> my_plot(g)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">In order to do this with a functional interface one could wrap it like this:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">>>> dg2 = lambda x: fprime(g, x, **options)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">>>> my_plot(dg2)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">If you like the one-liner that the function gives, you could call the Derivate class like this
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">>>> Derivate(g, **options)(x)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">Which is very similar to the functional way:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black">>>> fprime(g, x, **options)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">Another argument for having it as a class is that a function will be large and
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Georgia","serif";color:#202020">“<a href="http://mikeebert.tumblr.com/post/25998669005/large-functions-are-where-classes-go-to-hide"><span style="color:#202020;text-decoration:none">large functions
are where classes go to hide</span></a>”. This is a quote of Uncle Bob’s that we hear frequently in the third and fourth <a href="http://www.cleancoders.com/"><span style="color:#0F6F9F;text-decoration:none">Clean Coders</span></a> episodes. He states that
when a function starts to get big it’s most likely doing too much— a function should do one thing only and do that one thing well. Those extra responsibilities that we try to cram into a long function (aka method) can be extracted out into separate classes
or functions.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:black">The implementation in <a href="https://github.com/pbrod/numdifftools/blob/master/numdifftools/nd_cstep.py">
<span style="color:black">https://github.com/pbrod/numdifftools/blob/master/numdifftools/nd_cstep.py</span></a></span> is an attempt to do this.<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">For the use case where n>=1 and the Richardson/Romberg extrapolation method, I propose to factor this out in a separate class e.g. :<o:p></o:p></p>
<p class="MsoNormal">>>> class NDerivative(object):<o:p></o:p></p>
<p class="MsoNormal">…. def __init__(self, f, n=1, method=’central’, order=2, …**options):<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">It is very difficult to guarantee a certain accuracy for derivatives from finite differences. In order to get error-estimates for the derivatives one must do several functions evaluations. In my experience with numdifftools it is very difficult
to know exactly which step-size is best. Setting it too large or too small are equally bad and difficult to know in advance. Usually there is a very limited window of useful step-sizes which can be used for extrapolating the evaluated differences to a better
final result. The best step-size can often be found around (10*eps)**(1./s)*maximum(log1p(abs(x)), 0.1) where s depends on the method and derivative order. Thus one cannot improve the results indefinitely by adding more terms. With finite differences you
can hope the chosen sampling scheme gives you reasonable values and error-estimates, but many times, you just have to accept what you get. <o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Regarding the proposed API I wonder how useful the input arguments epsabs, epsrel will be?
<o:p></o:p></p>
<p class="MsoNormal">I also wonder how one can compute the outputs abserr_round, abserr_truncate accurately?
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Best regards<o:p></o:p></p>
<p class="MsoNormal" style="line-height:12.0pt"><b><span style="font-size:9.0pt;font-family:"Verdana","sans-serif";color:#1F497D">Per A. Brodtkorb<o:p></o:p></span></b></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<blockquote style="border:none;border-left:solid #CCCCCC 1.0pt;padding:0cm 0cm 0cm 6.0pt;margin-left:4.8pt;margin-right:0cm">
<div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
</div>
</blockquote>
<div>
<p class="MsoNormal"><span style="color:#1F497D">I am also putting a simple thought about possible API implementation for calculating derivative (gradient or jacobian), though this is a simple approach I was wondering about. Please do correct me if I overlooked
some important points. Waiting to hear from you.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;margin-left:11.25pt;text-indent:-18.0pt;line-height:14.65pt;mso-list:l4 level1 lfo6">
<![if !supportLists]><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D"><span style="mso-list:Ignore">1.<span style="font:7.0pt "Times New Roman"">
</span></span></span><![endif]><span style="font-size:10.0pt;font-family:"Arial Black","sans-serif";color:#1F497D">approx_fprime</span><span style="font-size:10.0pt;font-family:"Arial","sans-serif";color:#1F497D"><o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Georgia","serif";color:#1F497D">arguments</span><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D">: <em><span style="font-family:"Arial","sans-serif"">f</span></em> : <u>callable</u> function<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> <em><span style="font-family:"Arial","sans-serif"">x</span></em> : <u>ndarray</u> values at which the derivative needs to be calculated<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> <em><span style="font-family:"Arial","sans-serif"">method</span></em> : <u>str</u> method to be used to approximate the derivative - 'central', 'forward', 'backward',
'complex', 'richardson'<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> <em><span style="font-family:"Arial","sans-serif"">n</span></em> : <u>Integer</u> from 1 to 4 (Default 1) defining derivative order.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> <em><span style="font-family:"Arial","sans-serif"">order</span></em> : <u>Integer</u> from 1 to 4 (Default 2) defining order of basic method used. For 'central' methods,
it must be from the set [2,4].<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> <em><span style="font-family:"Arial","sans-serif"">args</span></em> : <u>tuple</u> arguments for the function f<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> <em><span style="font-family:"Arial","sans-serif"">kwargs</span></em> : <u>dict</u> Keyword arguments for function `f`.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<strong><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> </span></strong><em><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D">epsabs</span></em><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> : <u>float
or int optional</u> Absolute error tolerance.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> <em><span style="font-family:"Arial","sans-serif"">epsrel</span></em> : <u>float or int, optional</u> Relative error tolerance.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<strong><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> </span></strong><em><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D">disp</span></em><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> : <u>bool</u> Set
to True to print error messages.<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Georgia","serif";color:#1F497D">return</span><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> : <em><span style="font-family:"Arial","sans-serif"">res</span></em> : <u>DiffResult</u><o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> The differentiation result represented as a </span><tt><span style="font-size:10.0pt;color:#1F497D">DiffResult</span></tt><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> object. Important
attributes are:<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<tt><span style="font-size:10.0pt;color:#1F497D"> </span></tt><em><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D">x</span></em><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D">: <u>ndarray</u> solution
array,<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> <em><span style="font-family:"Arial","sans-serif"">success :</span></em> <u>bool</u> a flag indicating if the derivative was calculated successfully<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> <em><span style="font-family:"Arial","sans-serif"">message</span></em> : <u>str</u> which describes the cause of the error, if occurred<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> <em><span style="font-family:"Arial","sans-serif"">nfev </span></em>: <u>int</u> number of function evaluations<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> <em><span style="font-family:"Arial","sans-serif"">abserr_round </span></em> : <u>float</u> absolute value of the roundoff error, if applicable<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<em><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"> abserr_truncate </span></em><span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D">: <u>float</u> absolute value of the truncation error, if
applicable<o:p></o:p></span></p>
<p style="mso-margin-top-alt:0cm;margin-right:0cm;margin-bottom:7.5pt;margin-left:0cm;line-height:19.2pt">
<span style="font-size:9.0pt;font-family:"Arial","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="color:#1F497D">Cheers,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:#1F497D">Maniteja<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal">_______________________________________________<br>
SciPy-Dev mailing list<br>
<a href="mailto:SciPy-Dev@scipy.org" target="_blank">SciPy-Dev@scipy.org</a><br>
<a href="http://mail.scipy.org/mailman/listinfo/scipy-dev" target="_blank">http://mail.scipy.org/mailman/listinfo/scipy-dev</a><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
</div>
</div>
</div>
</div>
</body>
</html>