[SciPy-Dev] Regarding taking up project ideas and GSoC 2015

[Maniteja Nandana]

Hello everyone,

I am writing this mail to enquire about implementing numerical
differentiation package in scipy.

There have been discussions before (Issue #2035
<https://github.com/scipy/scipy/issues/2035>) and some PRs (PR #2835
<https://github.com/scipy/scipy/pull/2835>) to include tools to compute
derivatives in scipy.

According to the comments made in them, as far as I can understand, I see
that there are some ways to do derivatives on the computer with varying
 generality and accuracy ( by Alex Griffing
<https://github.com/scipy/scipy/issues/2035#issuecomment-23628615> ) :

   1. 1st order derivatives of special functions
   2. derivatives of univariate functions
   3. symbolic differentiation
   4. numerical derivatives - finite differences
   5. automatic or algorithmic differentiation

Clearly, as suggested in the thread, the 1st option is already done in
functions like *jv* and *jvp* in *scipy.special. *

I think everyone agreed that symbolic derivatives is out of scope of scipy.

Though I would like to hear more about the univariate functions.

Coming to finite differences, the modules described there, *statsmodels *and
*numdifftools, *they vary in aspects of speed and accuracy, in terms of
approaches followed as mentioned in Joseph Perktold comment
<https://github.com/scipy/scipy/pull/2835#issuecomment-52372036>

   - *Statsmodels *used complex step derivatives, which are for first order
   derivatives and have only truncation error, no roundoff error since
   there is no subtraction.
   - *Numdifftools *uses adaptive step-size to calculate finite
   differences, but will suffer from dilemma to choose small step-size to
   reduce truncation error but at the same time avoid subtractive cancellation
   at too small values

I have read the papers used by both the implementations:
 *Statsmodels *Statistical applications of the complex-step method of
numerical differentiation, Ridout, M.S.
<https://drive.google.com/file/d/0BwUeCS0FJLRucXdCc1JOTEY0cGc/view?usp=sharing>
 *Numdifftools *The pdf attached in the github repository DERIVEST.pdf
<https://drive.google.com/file/d/0BwUeCS0FJLRuYW1MNlp2enJCaHM/view?usp=sharing>

Just pointing out in this platform, I think there is an error in equation
13 in DERIVEST, It should be

f'-0() = 2f'-delta/2() - f'-delta(),  instead of f'-0() = 2f'-delta() -
f'-delta/2()

as also correctly mentioned in the matlab code that followed the equation

As much as my understanding from the discussions goes, the statsmodels
implementation uses elegant broadcasting. Though I get the idea seeing the
code, I would really appreciate some examples that clearly explain this.

Also the complex-step method is only for first order derivatives and that
function is analytic, so that Cauchy-Riemann equations are valid. So, is it
possible to differentiate any function with this ?

Also as I was discussing with Christoph Deil, the API implementation issue
of whether to use classes, as in numdifftools or as functions, as in
statsmodels came to the fore. Though I am not an expert in it, I would love
to hear some suggestions on it.

Though at this point AD seems ahead of time, it is powerful in forward and
reverse methods, moreover complex-step is somewhat similar to it. The
packages *ad *and *algopy *use AD. Also, there were concerns with
interfacing these methods with C/ Fortran functions. It would also be great
if there could be suggestions regarding whether to implement these methods.

At the same time, it would be really helpful if any new methods or packages
to be looked into could be suggested.

Waiting in anticipation for your feedback and response. Happy to learn :)
Thanks for reading along my lengthy mail. Please do correct if I did some
mistake.

I have attached the documents I have related to these issues, most
importantly *The Complex-Step Derivative Approximation by **JOAQUIM R. R.
A. MARTINS*

*Numerical differentiation
<https://drive.google.com/folderview?id=0BwUeCS0FJLRufnJaVko3MGpJX0Nud3R0dHgyc2JBYUgxVkhBTkNvbkhFQWZucmhWSzlaVVk&usp=sharing>*

Cheers,
Maniteja.
_______________________________________________
SciPy-Dev mailing list
SciPy-Dev at scipy.org
http://mail.scipy.org/mailman/listinfo/scipy-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150306/53b4549e/attachment.html>