Hi all, Last week we had a core developer meeting. In attendance were Stéfan van der Walt, Alexandre Fioravante de Siqueira, Gregory Lee, and myself. This is probably the last non-public core developer meeting. Next year we will open them up to anyone, like NumPy does. Look out for announcements on this list. In the meantime, in the interest of transparency, I'm posting a summary of the meeting based on my notes. 1. Performance with Cython, Numba, GPU/Cuda/CuPy We discussed the performance of scikit-image, which as we know is generally sluggish compared to closer-to-the-metal, optimised libraries such as OpenCV, not to mention GPU-enabled tools. Although we value [1] clean, clear implementations over cutting edge performance, in many cases, the gap is 20-30x, not a more acceptable 20-30%, or even 2x. We do get a significant number of support requests about this, so we should pay attention to it. Among the speedup strategies discussed were: - Cythonising all the things - Numba (a NumPy-focused just-in-time compiler) - CuPy (a drop-in NumPy replacement that computes on nvidia GPUs) Stéfan noted, and we all agreed, that we don't have the resources to maintain parallel "backends" that depend on the users' hardware, GPU drivers, and optional dependencies, so we are generally limited to tools that give us speedups "for free". As such, since we can't limit ourselves to only accepting CuPy arrays as input, CuPy is not currently an option. However, it's plausible that we could soon get CuPy support for free thanks to the NEP 18 [2] NumPy proposal, which would allow functions such as `np.sum` to be executed differently depending on the type of its arguments, so that `np.sum` on a dask array would return a dask graph node, while it would execute on the GPU if its argument was a CuPy array. Thus, scikit-image functions composed mostly of high-level NumPy calls will be accelerated automatically. We will be monitoring developments in this space very closely to see how they can be applied to scikit-image. Cythonising all of scikit-image (even outside of the inner loops) has a high cost for readability, debugging, and building, since our entire code base would then be opaque to most Python tools, so this is not an option we will pursue. Generally, also, where we have run profiling, scikit-image functions spend over 80% of their time in NumPy operations or already-Cythonised code, so this approach would at best give us a 20% speedup in most cases. (Other libraries have obtained 2-3x speedups [3], but that is in a very different domain.) The other option that was mentioned is Numba. Here we could perhaps make a nearly immediate push. As some of you might know by now, I received a fellowship from the Chan-Zuckerberg Initiative to improve the scikit-image library and the surrounding ecosystem for microscopy and other bioimage analysis. (I will write more about this separately!) This is the first dedicated funding that the scikit-image project has received, and we will continue pushing to ensure that it's only the beginning. More specifically, Stéfan pointed out that the Moore Foundation is aiming to fund more open source in the scientific Python space in the near future. We are planning to submit both a dedicated scikit-image proposal, and a more specific joint proposal between scikit-image and Numba. 2. Proposal for our vision/mission/values statement. Already linked above is my pull request to add a vision/mission/values statement for scikit-image to our documentation [1]. We'd like to get broad input before merging that (work-in-progress) PR. Stéfan pointed out that it's very easy to get into a "refine" mode when reading a document, fiddling with the grammar and phrasing of things, or even critiquing the statements that are already there. It's much harder to think of what's *missing* from the document. Is there something you think such this document should say that it doesn't? Please get involved by commenting on the pull request! 3. Discussion of existing pull requests We discussed several pull requests currently in progress: - in the pull request adding fused type support to denoising and warping [4], which includes work by Matthew Bowden, Mark Harfouche, and myself, Mark pointed out that we have a *comment* in our existing code that our biquadratic interpolation implementation is incorrect, but it nevertheless gets executed if requested. We agreed that this should `transform.warp` should instead raise a `NotImplementedError` if `order=2` is given. - Alexandre's `threshold_multiotsu` pull request [5] is currently stalled due to the tension between performance and elegant code. The implementation that we all agree makes sense here uses `itertools.combinations` instead of multiply-nested for-loops. However, even if one ignores the massive advantage of Cython in for-loops, the pure Python for-loop version is much faster than the itertools version. I suspect the optimal way to do this is an array-based implementation of itertools.combinations in Cython, but it's tricky for sure. Any help with optimising that code is much appreciated! - Greg pointed out that some parts of the library use `img_as_float` in conjunction with `np.clip`. This used to be fine until someone [6] came around and made `img_as_float` pass through float32 arrays without checking that their range. So now it was possible to send in a float32 image in an arbitrary range and then have it be clipped to [0, 1]. Greg is fixing this behaviour for denoising in #3584 [7], but a more complete audit will need to happen. A good place for discussion is probably #3373 [8]. - Finally, we discussed Mark's demosaicing pull request [9], and decided that it does indeed make sense to have a reference implementation in scikit-image, probably within the io module. That's it for my summary. Stéfan, Greg, and Alexandre, if I missed something, please reply to this list! See you all in 2019 for our first public meeting! =) Juan. .. [1]: https://github.com/scikit-image/scikit-image/pull/3585 .. [2]: https://www.numpy.org/neps/nep-0018-array-function-protocol.html .. [3]: http://matthewrocklin.com/blog/work/2014/05/01/Fast-Data-Structures .. [4]: https://github.com/scikit-image/scikit-image/pull/3486 .. [5]: https://github.com/scikit-image/scikit-image/pull/3385 .. [6]: https://github.com/scikit-image/scikit-image/pull/3052 .. [7]: https://github.com/scikit-image/scikit-image/pull/3584 .. [8]: https://github.com/scikit-image/scikit-image/issues/3373 .. [9]: https://github.com/scikit-image/scikit-image/pull/3057