Mailman 3 March 2020 - SciPy-Dev

welcome Lucas Roberts to the SciPy core team
by Ralf Gommers March 16, 2020

March 16, 2020

Hi all, On behalf of the SciPy developers I'd like to welcome Lucas Roberts as a member of the core team. Lucas has been contributing for well over a year, making nice improvements and fixes to scipy.stats and scipy.special. He has also done a lot of valuable reviews of stats PRs. Here is an overview of his SciPy PRs: https://github.com/scipy/scipy/pulls/rlucas7 I'm looking to Lucas' continued contributions! Cheers, Ralf

5 4

Multithreading sparse matrix to dense vector(s) multiplication
by Jorge Guerra March 16, 2020

March 16, 2020

Hello everyone, I'm quite new to the SciPy development community. I've identified a potential improvement that would be very helpful: in scipy/sparse/sparsetools/csr.h @ lines 1092 and 1129 in the csr_matvec() and mat_vecs() methods I would like to parallelize the outer for loop over the rows of the input sparse matrix. The current serial code looks just about optimal, but should take advantage of a multicore machine. This is a place where SciPy is deficient. Going through the compilation, SciPy depends on pThreads. However, my proposed change would be trivial by including OpenMP, but I don't necessarily want to add to the overall dependencies. Would anyone more familiar with pThreads be willing to collaborate with me to accomplish this proposed task? Or perhaps advice on an alternative to get multithreaded sparse matrix to dense vector multiplication implemented? Thank you all! Jorge -- *Jorge E. Guerra, PhD.* *Research Scientist, CIMMS/NOAA* *Earth Systems Research Laboratory, Boulder, CO*

4 4

Degraded performance via Cython LAPACK wrappers
by Ilhan Polat March 15, 2020

March 15, 2020

Dear all, I am trying to increase the performance of a few functions in linalg module by cythonizing trivial but time-consuming tasks. After not getting too much performance, I decided to consult the scipy folks about it. I have the following cython MVE specifically working on fortran contiguous arrays for simplicity. @cython.boundscheck(False) @cython.wraparound(False) @cython.cdivision(True) cpdef void zzz(double[::1, :] a, double[::1, :]b) nogil: cdef int n, nrhs n = a.shape[0] nrhs = b.shape[1] lda = n cdef int *ipiv = <int *>malloc(<int>n * sizeof(int)) cdef int info = 0 if not ipiv: raise MemoryError() # dgesv(int *n, int *nrhs, d *a, int *lda, int *ipiv, d *b, int *ldb, int *info) dgesv(&n, &nrhs, &a[0,0], &lda, &ipiv[0], &b[0,0], &lda, &info) free(ipiv) This is supposed to be much faster than the regular sp.linalg.solve(a, b) call since it bypasses a lot of checks and the relevant lapack flavor detection etc. However using the following data n = 150 a = np.empty([n, n], dtype=float, order='F') a[:, :] = np.random.rand(n, n) + np.eye(n)*5 b = np.asfortranarray(np.random.rand(n, 1)) I see that for about 800 > n > 50 the results are almost identical which is pretty surprising as cython code should be spending a lot of time somewhere. I haven't been successful enabling the line trace in cython code hence can anyone please tell me where I am losing performance and how I can remedy it? Best,

1 1

Medfilt with signal size less than kernel_size
by Władysław Raczek March 15, 2020

March 15, 2020

Hi everyone, I'm new contributor here. Recently I've looked into issue https://github.com/scipy/scipy/issues/11503, and it turned out everything was fine (e.g. according to documentation), but then grlee77 suggested that maybe it's worth raising UserWarning for such cases. What do you think? It doesn't look so complicated, and I would be happy to do it as my first contribution. Regards, Vladyslav Rachek

2 1

Welcome Peter Larson to Core Team!
by Tyler Reddy March 10, 2020

March 10, 2020

I'm pleased to announce the addition of Peter Larson ( https://github.com/pmla ) to the SciPy Core Developer team! Welcome! Best wishes, Tyler

7 6

changing default for median_absolute_deviation
by Lucas Roberts March 8, 2020

March 8, 2020

Hi scipy-dev, Looking for guidance if the change warrants ignoring deprecation policy or if I should create a deprecation warning for existing behavior to change defaults in median_absolute_deviation() PR: https://github.com/scipy/scipy/pull/11431 [CONTEXT] In this PR: https://github.com/scipy/scipy/pull/9637 the median_absolute_deviation() function was added to scipy.stats the function takes in a scale argument that allows for a robust estimator of the scale (robust to outliers). The choice of scale constant assuming normal input data seemed reasonable to me at the time. However, in this PR: https://github.com/scipy/scipy/issues/11090 a few thought otherwise and we came to the conclusion that the default scale should be 1 which would also ensure internal consistency with the stats.iqr() function (similar signature and functionality). I've opened a PR to change the defaults here: https://github.com/scipy/scipy/pull/11431 and pinged those on the 11090 issue. [ON DEPRECATION]: [REASON FOR] Main reason to not follow deprecation policy are: 1. Following deprecation policy would maintain existing confusing behavior for up to 1 year from now. 2. Function released in 1.3.0 so somewhat new 3. Seems the default is confusing users and is non-obvious and inconsistent with iqr() default [REASON NOT] 1. Some users may depend on existing default 2. Defaults with normal scaling exist in several places (c.f. 11090 issue comments) 3. Deprecation warning needs to be done to give fair warning of the change. [COMMENTS] I haven't much experience here with deprecations, when to do it vs consider a defect so would appreciate any guidance. If we think the deprecation cycle should follow, should I leave the PR and open a separate deprecation PR? https://github.com/scipy/scipy/pull/11431 Alternately could revert changes and convert to a deprecation warning PR. Thanks in advance. -- -Lucas

2 1

What for the new contributor?
by Władysław Raczek March 3, 2020

March 3, 2020

Hi everybody, I'm third year student on Theoretical Computer Science. This year we have to contribute to open-source project, and I've chosen SciPy because it seems to be interesting, and it also seems to have lot of things to do. I've looked rather briefly through the list of issues, and I've come to thought that each issue is either a documentation issue, or some pretty complicated stuff. Maybe someone here has any thoughts on how can a new person in project (like me) contribute, e.g. You have some issues in mind, or You yourself opened some issue You consider suitable for a new contributor, and it didn't get a PR yet, or smth. else :) Would be truly grateful for help! Regards, Vladyslav Rachek P.S. This isn't about being afraid of doing "complicated stuff" but rather about help with suggesting about how to get to that "stuff" from beginner's level, doing useful things on the way :)

2 1

Review of PR 11119 and 10796
by Christoph Baumgarten March 1, 2020

March 1, 2020

Hi, two of my PRs have been open for a while, and it would be great if someone has time for a review: Cramer-von-Mises test: https://github.com/scipy/scipy/pull/11119 Exact p-values of the Wilcoxon test: https://github.com/scipy/scipy/pull/10796 Thanks Christoph

2 1

Disable rpaths in shared objects when cross compiling
by Greg Anders March 1, 2020

March 1, 2020

Hi all, I'm cross compiling Scipy in Yocto Linux for an embedded platform. I'm able to compile Scipy, but Yocto is giving me warnings because the shared objects are linked with the -rpath flag using absolute paths on the build host. I'd like to disable the -rpath flag when linking the shared objects, but I'm not sure how to do that. I've dug through the numpy.distutils code and I found the following in the CCompiler_customize_cmd function: if allow('rpath'): self.set_runtime_library_dirs(cmd.rpath) So it *looks* like if I can pass 'rpath' into the optional `ignore` paramater to the `customize_cmd` function, numpy will not use the -rpath flag when linking. However, it's not clear to me how to _use_ the `ignore` parameter of `customize_cmd`. All usages of `customize_cmd` in build_ext.py only use a single parameter, so `ignore` is always set to the default value of an empty tuple. Is there an easier way to disable rpaths than patching numpy.distutils to path `ignore=('rpath')` as a second parameter to `customize_cmd`? Thanks! Greg

2 1