[SciPy-Dev] Regarding taking up project ideas and GSoC 2015

[Ralf Gommers]

Hi Maniteja,


On Fri, Mar 6, 2015 at 1:12 PM, Maniteja Nandana <
maniteja.modesty067 at gmail.com> wrote:

> Hello everyone,
>
> I am writing this mail to enquire about implementing numerical
> differentiation package in scipy.
>
> There have been discussions before (Issue #2035
> <https://github.com/scipy/scipy/issues/2035>) and some PRs (PR #2835
> <https://github.com/scipy/scipy/pull/2835>) to include tools to compute
> derivatives in scipy.
>
> According to the comments made in them, as far as I can understand, I see
> that there are some ways to do derivatives on the computer with varying
>  generality and accuracy ( by Alex Griffing
> <https://github.com/scipy/scipy/issues/2035#issuecomment-23628615> ) :
>
>    1. 1st order derivatives of special functions
>    2. derivatives of univariate functions
>    3. symbolic differentiation
>    4. numerical derivatives - finite differences
>    5. automatic or algorithmic differentiation
>
> Clearly, as suggested in the thread, the 1st option is already done in
> functions like *jv* and *jvp* in *scipy.special. *
>
> I think everyone agreed that symbolic derivatives is out of scope of
> scipy.
>

Definitely, symbolic anything is out of scipy for scipy:)


> Though I would like to hear more about the univariate functions.
>
> Coming to finite differences, the modules described there, *statsmodels *and
> *numdifftools, *they vary in aspects of speed and accuracy, in terms of
> approaches followed as mentioned in Joseph Perktold comment
> <https://github.com/scipy/scipy/pull/2835#issuecomment-52372036>
>
>    - *Statsmodels *used complex step derivatives, which are for first
>    order derivatives and have only truncation error, no roundoff error
>    since there is no subtraction.
>    - *Numdifftools *uses adaptive step-size to calculate finite
>    differences, but will suffer from dilemma to choose small step-size to
>    reduce truncation error but at the same time avoid subtractive cancellation
>    at too small values
>
> I have read the papers used by both the implementations:
>  *Statsmodels *Statistical applications of the complex-step method of
> numerical differentiation, Ridout, M.S.
> <https://drive.google.com/file/d/0BwUeCS0FJLRucXdCc1JOTEY0cGc/view?usp=sharing>
>  *Numdifftools *The pdf attached in the github repository DERIVEST.pdf
> <https://drive.google.com/file/d/0BwUeCS0FJLRuYW1MNlp2enJCaHM/view?usp=sharing>
>
> Just pointing out in this platform, I think there is an error in equation
> 13 in DERIVEST, It should be
>
> f'-0() = 2f'-delta/2() - f'-delta(),  instead of f'-0() = 2f'-delta() -
> f'-delta/2()
>
> as also correctly mentioned in the matlab code that followed the equation
>

You may want to let the author know, he'll probably appreciate it.


> As much as my understanding from the discussions goes, the statsmodels
> implementation uses elegant broadcasting. Though I get the idea seeing the
> code, I would really appreciate some examples that clearly explain this.
>
> Also the complex-step method is only for first order derivatives and that
> function is analytic, so that Cauchy-Riemann equations are valid. So, is it
> possible to differentiate any function with this ?
>
> Also as I was discussing with Christoph Deil, the API implementation issue
> of whether to use classes, as in numdifftools or as functions, as in
> statsmodels came to the fore. Though I am not an expert in it, I would love
> to hear some suggestions on it.
>

It will be important to settle on a clean API. There's no general
preference for classes or functions in Scipy, the pros/cons have to be
looked at in detail for this functionality. The scope of the scipy.diff
project is quite large, so starting a document (as I think you've already
discussed with Christoph?) outlining the API that can be reviewed will be a
lot more efficient than trying to do it by email alone.


> Though at this point AD seems ahead of time, it is powerful in forward and
> reverse methods, moreover complex-step is somewhat similar to it. The
> packages *ad *and *algopy *use AD. Also, there were concerns with
> interfacing these methods with C/ Fortran functions. It would also be
> great if there could be suggestions regarding whether to implement these
> methods.
>

It's been around for a while so not sure about "ahead of its time", but yes
it can be powerful. It's a large topic though, should be out of scope for
this GSoC project. Good finite difference methods will be challenging
enough:) That doesn't mean that AD is out of scope for Scipy necessarily,
but that's for another time to discuss.

At the same time, it would be really helpful if any new methods or packages
> to be looked into could be suggested.
>

I think what's in numdifftools and statsmodels is a good base to build on.
What could be very useful in addition though is an indepent reference
implementation of the methods you're working on. This could be
Matlab/R/Julia functions or some package written by the author of a paper
you're using. I don't have concrete suggestions now - you have a large
collection of papers - but you could already check the papers you're using.

Cheers,
Ralf


> Waiting in anticipation for your feedback and response. Happy to learn :)
> Thanks for reading along my lengthy mail. Please do correct if I did some
> mistake.
>
> I have attached the documents I have related to these issues, most
> importantly *The Complex-Step Derivative Approximation by **JOAQUIM R. R.
> A. MARTINS*
>
> *Numerical differentiation
> <https://drive.google.com/folderview?id=0BwUeCS0FJLRufnJaVko3MGpJX0Nud3R0dHgyc2JBYUgxVkhBTkNvbkhFQWZucmhWSzlaVVk&usp=sharing>*
>
> Cheers,
> Maniteja.
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150309/6e09704d/attachment.html>