[SciPy-Dev] Regarding taking up project ideas and GSoC 2015

Sat Mar 14 11:53:47 EDT 2015

Hi everyone,

I was hoping if I could get some suggestions regarding the API for
*scipy.diff* package.

   1. Type of input to be given - callable function objects or a set of
   points as in scipy.integrate.
   2. Parameters to be given to derivative methods, like *method *(as in
   scipy.optimize) to accommodate options like *central, forward, backward,
   complex or richardson.*
   3. The maximum order of derivative needed ? Also the values of order *k*
   used in the basic method to determine the truncation error O(h^k) ?
   4. API defined in terms of functions(as in statsmodels) or classes(as in
   numdifftools) ?
   5. Return type of the methods should contain the details of the result,
   like *error *?( on lines of OptimizeResult, as in scipy.optimize )

I would really appreciate some feedback and suggestions on these issues.
The whole draft of the proposal can be seen here
<https://github.com/maniteja123/GSoC/wiki/Proposal%3A-add-finite-difference-numerical-derivatives-as-%60%60scipy.diff%60%60>
.

Thanks for reading along and giving your valuable inputs.

Cheers,
Maniteja.

On Wed, Mar 11, 2015 at 11:44 PM, Maniteja Nandana <
maniteja.modesty067 at gmail.com> wrote:

> Hi everyone,
>
> I have created a Wiki <https://github.com/maniteja123/GSoC/wiki/> page
> and draft proposal
> <https://github.com/maniteja123/GSoC/wiki/Proposal:-add-finite-difference-numerical-derivatives-as-%60%60scipy.diff%60%60> regarding
> some approaches for API implementation for scipy.diff package after
> discussing with Christoph Deil. I would really appreciate some feedback and
> suggestions to incorporate more sound and concrete ideas into the proposal.
> I also wanted to ask if it would be better to start a wiki page regarding
> this on scipy repository. I thought it would be better to do so once the
> proposal is more concrete.
>
> Thanks again for reading along my proposal and waiting in anticipation for
> your suggestions.
>
> Cheers,
> Maniteja
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
> On Mon, Mar 9, 2015 at 12:23 PM, Ralf Gommers <ralf.gommers at gmail.com>
> wrote:
>
>> Hi Maniteja,
>>
>>
>> On Fri, Mar 6, 2015 at 1:12 PM, Maniteja Nandana <
>> maniteja.modesty067 at gmail.com> wrote:
>>
>>> Hello everyone,
>>>
>>> I am writing this mail to enquire about implementing numerical
>>> differentiation package in scipy.
>>>
>>> There have been discussions before (Issue #2035
>>> <https://github.com/scipy/scipy/issues/2035>) and some PRs (PR #2835
>>> <https://github.com/scipy/scipy/pull/2835>) to include tools to compute
>>> derivatives in scipy.
>>>
>>> According to the comments made in them, as far as I can understand, I
>>> see that there are some ways to do derivatives on the computer with varying
>>>  generality and accuracy ( by Alex Griffing
>>> <https://github.com/scipy/scipy/issues/2035#issuecomment-23628615> ) :
>>>
>>>    1. 1st order derivatives of special functions
>>>    2. derivatives of univariate functions
>>>    3. symbolic differentiation
>>>    4. numerical derivatives - finite differences
>>>    5. automatic or algorithmic differentiation
>>>
>>> Clearly, as suggested in the thread, the 1st option is already done in
>>> functions like *jv* and *jvp* in *scipy.special. *
>>>
>>> I think everyone agreed that symbolic derivatives is out of scope of
>>> scipy.
>>>
>>
>> Definitely, symbolic anything is out of scipy for scipy:)
>>
>>
>>> Though I would like to hear more about the univariate functions.
>>>
>>> Coming to finite differences, the modules described there, *statsmodels
>>> *and *numdifftools, *they vary in aspects of speed and accuracy, in
>>> terms of approaches followed as mentioned in Joseph Perktold comment
>>> <https://github.com/scipy/scipy/pull/2835#issuecomment-52372036>
>>>
>>>    - *Statsmodels *used complex step derivatives, which are for first
>>>    order derivatives and have only truncation error, no roundoff error
>>>    since there is no subtraction.
>>>    - *Numdifftools *uses adaptive step-size to calculate finite
>>>    differences, but will suffer from dilemma to choose small step-size to
>>>    reduce truncation error but at the same time avoid subtractive cancellation
>>>    at too small values
>>>
>>> I have read the papers used by both the implementations:
>>>  *Statsmodels *Statistical applications of the complex-step method of
>>> numerical differentiation, Ridout, M.S.
>>> <https://drive.google.com/file/d/0BwUeCS0FJLRucXdCc1JOTEY0cGc/view?usp=sharing>
>>>  *Numdifftools *The pdf attached in the github repository DERIVEST.pdf
>>> <https://drive.google.com/file/d/0BwUeCS0FJLRuYW1MNlp2enJCaHM/view?usp=sharing>
>>>
>>> Just pointing out in this platform, I think there is an error in
>>> equation 13 in DERIVEST, It should be
>>>
>>> f'-0() = 2f'-delta/2() - f'-delta(),  instead of f'-0() = 2f'-delta() -
>>> f'-delta/2()
>>>
>>> as also correctly mentioned in the matlab code that followed the equation
>>>
>>
>> You may want to let the author know, he'll probably appreciate it.
>>
>>
>>> As much as my understanding from the discussions goes, the statsmodels
>>> implementation uses elegant broadcasting. Though I get the idea seeing the
>>> code, I would really appreciate some examples that clearly explain this.
>>>
>>> Also the complex-step method is only for first order derivatives and
>>> that function is analytic, so that Cauchy-Riemann equations are valid. So,
>>> is it possible to differentiate any function with this ?
>>>
>>> Also as I was discussing with Christoph Deil, the API implementation
>>> issue of whether to use classes, as in numdifftools or as functions, as in
>>> statsmodels came to the fore. Though I am not an expert in it, I would love
>>> to hear some suggestions on it.
>>>
>>
>> It will be important to settle on a clean API. There's no general
>> preference for classes or functions in Scipy, the pros/cons have to be
>> looked at in detail for this functionality. The scope of the scipy.diff
>> project is quite large, so starting a document (as I think you've already
>> discussed with Christoph?) outlining the API that can be reviewed will be a
>> lot more efficient than trying to do it by email alone.
>>
>>
>>> Though at this point AD seems ahead of time, it is powerful in forward
>>> and reverse methods, moreover complex-step is somewhat similar to it. The
>>> packages *ad *and *algopy *use AD. Also, there were concerns with
>>> interfacing these methods with C/ Fortran functions. It would also be
>>> great if there could be suggestions regarding whether to implement these
>>> methods.
>>>
>>
>> It's been around for a while so not sure about "ahead of its time", but
>> yes it can be powerful. It's a large topic though, should be out of scope
>> for this GSoC project. Good finite difference methods will be challenging
>> enough:) That doesn't mean that AD is out of scope for Scipy necessarily,
>> but that's for another time to discuss.
>>
>> At the same time, it would be really helpful if any new methods or
>>> packages to be looked into could be suggested.
>>>
>>
>> I think what's in numdifftools and statsmodels is a good base to build
>> on. What could be very useful in addition though is an indepent reference
>> implementation of the methods you're working on. This could be
>> Matlab/R/Julia functions or some package written by the author of a paper
>> you're using. I don't have concrete suggestions now - you have a large
>> collection of papers - but you could already check the papers you're using.
>>
>> Cheers,
>> Ralf
>>
>>
>>> Waiting in anticipation for your feedback and response. Happy to learn :)
>>> Thanks for reading along my lengthy mail. Please do correct if I did
>>> some mistake.
>>>
>>> I have attached the documents I have related to these issues, most
>>> importantly *The Complex-Step Derivative Approximation by **JOAQUIM R.
>>> R. A. MARTINS*
>>>
>>> *Numerical differentiation
>>> <https://drive.google.com/folderview?id=0BwUeCS0FJLRufnJaVko3MGpJX0Nud3R0dHgyc2JBYUgxVkhBTkNvbkhFQWZucmhWSzlaVVk&usp=sharing>*
>>>
>>> Cheers,
>>> Maniteja.
>>> _______________________________________________
>>> SciPy-Dev mailing list
>>> SciPy-Dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>>
>>
>>
>> _______________________________________________
>> SciPy-Dev mailing list
>> SciPy-Dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20150314/adbb70fe/attachment.html>