[SciPy-Dev] scipy.special.smirnov, scipy/special/cephes/kolmogorov.c

[Paul van Mulbregt]

Hi,
Re: scipy.special.smirnov, scipy/special/cephes/kolmogorov.c 

While looking at some problems I encountered with the KSone statistics distribution, I delved deep into the scipy.special functions smirnov() and smirnovi() in kolmogorov.c and came across some accuracy and convergence issues.  https://github.com/scipy/scipy/issues/7456 <https://github.com/scipy/scipy/issues/7456>

I have a stab at modifications which would address them but the issues turned out to be much more subtle/interesting than I was expecting.

1. The first part involves modifying algorithms and C code in scipy/special/cephes/kolmogorov.c.
The cephes code appears to have undergone mostly maintenance changes the last N years. 
Is there some documentation on policies for the cephes C code? 
Such as:
* What kind of changes are being accepted?
* When to report OVERFLOW or UNDERFLOW?
* How much commenting is appropriate for algorithm changes?
* Is it tabs or spaces in the C code? (Both styles appear to be in use.)
* Are there specs, either explicit or implicit, for the functions in cephes? 
* What is required for an algorithmic change to be considered?
* What is required for a new function to be added?
* Is the test suite expansive enough to detect newly introduced errors?

[Don't know if it is appropriate for this list, but if anyone is willing to take a look at the mods as they stand and offer comments before I’ve gone way far in a wrong direction, that would be welcomed. https://github.com/pvanmulbregt/scipy/tree/smirnov_stability <https://github.com/pvanmulbregt/scipy/tree/smirnov_stability> ]

2. The second part of the change would involve exporting a newly created C function smirnovp(), the derivative of smirnov(), to be used in the ksone python code.
Is there documentation on what would be required to export a new cephes function?
So far I’ve found that I needed to modify a dozen files just to get my installation to compile and pass tests.
scipy/special/__init__.py
scipy/special/_legacy.pxd
scipy/special/_ufuncs.pyx
scipy/special/_ufuncs_defs.h
scipy/special/add_newdocs.py
scipy/special/cephes.h
scipy/special/cephes/cephes_names.h
scipy/special/cython_special.pxd
scipy/special/cython_special.pyx
scipy/special/generate_ufuncs.py
scipy/special/tests/test_basic.py
scipy/special/tests/test_cython_special.py

That suggest that the process is non-trivial and that there must be even more.  Can someone point me to the place(s) in the doc where this is described?


Thanks!

Paul

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20170605/d0117a3a/attachment.html>