Mailman 3 July 2019 - SciPy-Dev

NumPy 1.17.0rc1 released
by Charles R Harris July 26, 2019

July 26, 2019

Hi All, On behalf of the NumPy team I am pleased to announce the release of NumPy 1.17.0rc1. The 1.17 release contains a number of new features that should substantially improve its performance and usefulness. The Python versions supported are 3.5-3.7, note that Python 2.7 has been dropped. Python 3.8b1 should work with the released source packages, but there are no guarantees about future releases. Highlights of this release are: - A new extensible random module along with four selectable random numbe5 generators and improved seeding designed for use in parallel processes has been added. The currently available bit generators are MT19937, PCG64, Philox, and SFC64. - NumPy's FFT implementation was changed from fftpack to pocketfft, resulting in faster, more accurate transforms and better handling of datasets of prime length. - New radix sort and timsort sorting methods. It is currently not possible to choose which will be used, but they are hardwired to the datatype and used when either ``stable`` or ``mergesort`` is passed as the method. - Overriding numpy functions is now possible by default Downstream developers should use Cython >= 0.29.10 for Python 3.8 support and OpenBLAS >= 3.7 (not currently out) to avoid problems on the Skylake architecture. The NumPy wheels on PyPI are built from the OpenBLAS development branch in order to avoid those problems. Wheels for this release can be downloaded from PyPI <https://pypi.org/project/numpy/1.17.0rc1/>, source archives and release notes are available from Github <https://github.com/numpy/numpy/releases/tag/v1.17.0rc1>. *Contributors* A total of 142 people contributed to this release. People with a "+" by their names contributed a patch for the first time. - Aaron Voelker + - Abdur Rehman + - Abdur-Rahmaan Janhangeer + - Abhinav Sagar + - Adam J. Stewart + - Adam Orr + - Albert Thomas + - Alex Watt + - Alexander Blinne + - Alexander Shadchin - Allan Haldane - Ander Ustarroz + - Andras Deak - Andreas Schwab - Andrew Naguib + - Andy Scholand + - Ankit Shukla + - Anthony Sottile - Antoine Pitrou - Antony Lee - Arcesio Castaneda Medina + - Assem + - Bernardt Duvenhage + - Bharat Raghunathan + - Bharat123rox + - Bran + - Bruce Merry + - Charles Harris - Chirag Nighut + - Christoph Gohlke - Christopher Whelan + - Chuanzhu Xu + - Daniel Hrisca - Daniel Lawrence + - Debsankha Manik + - Dennis Zollo + - Dieter Werthmüller + - Dominic Jack + - EelcoPeacs + - Eric Larson - Eric Wieser - Fabrice Fontaine + - Gary Gurlaskie + - Gregory Lee + - Gregory R. Lee - Hameer Abbasi - Haoyu Sun + - He Jia + - Hunter Damron + - Ian Sanders + - Ilja + - Isaac Virshup + - Isaiah Norton + - Jaime Fernandez - Jakub Wilk - Jan S. (Milania1) + - Jarrod Millman - Javier Dehesa + - Jeremy Lay + - Jim Turner + - Jingbei Li + - Joachim Hereth + - John Belmonte + - John Kirkham - John Law + - Jonas Jensen - Joseph Fox-Rabinovitz - Joseph Martinot-Lagarde - Josh Wilson - Juan Luis Cano Rodríguez - Julian Taylor - Jérémie du Boisberranger + - Kai Striega + - Katharine Hyatt + - Kevin Sheppard - Kexuan Sun - Kiko Correoso + - Kriti Singh + - Lars Grueter + - Maksim Shabunin + - Manvi07 + - Mark Harfouche - Marten van Kerkwijk - Martin Reinecke + - Matthew Brett - Matthias Bussonnier - Matti Picus - Michel Fruchart + - Mike Lui + - Mike Taves + - Min ho Kim + - Mircea Akos Bruma - Nick Minkyu Lee - Nick Papior - Nick R. Papior + - Nicola Soranzo + - Nimish Telang + - OBATA Akio + - Oleksandr Pavlyk - Ori Broda + - Paul Ivanov - Pauli Virtanen - Peter Andreas Entschev + - Peter Bell + - Pierre de Buyl - Piyush Jaipuriayar + - Prithvi MK + - Raghuveer Devulapalli + - Ralf Gommers - Richard Harris + - Rishabh Chakrabarti + - Riya Sharma + - Robert Kern - Roman Yurchak - Ryan Levy + - Sebastian Berg - Sergei Lebedev + - Shekhar Prasad Rajak + - Stefan van der Walt - Stephan Hoyer - SuryaChand P + - Søren Rasmussen + - Thibault Hallouin + - Thomas A Caswell - Tobias Uelwer + - Tony LaTorre + - Toshiki Kataoka - Tyler Moncur + - Tyler Reddy - Valentin Haenel - Vrinda Narayan + - Warren Weckesser - Weitang Li - Wojtek Ruszczewski - Yu Feng - Yu Kobayashi + - Yury Kirienko + - @aashuli + - @euronion + - @luzpaz - @parul + - @spacescientist + Cheers, Charles Harris

1 2

PR for the addition of scipy.stats.mgc
by Sambit Panda July 26, 2019

July 26, 2019

Hello all, For the past few weeks, I have been working on integrating MGC into scipy.stats as an additional “correlation function” (https://github.com/scipy/scipy/pull/10524). Just for reference, I had sent previous messages about this here: https://mail.python.org/pipermail/scipy-dev/2019-June/023544.html and https://mail.python.org/pipermail/scipy-dev/2019-June/023566.html. For the time being, I have only added MGC and not any of the other implementations (will do this during the next few weeks). I followed the SciPy development workflow instructions and invite your review. Thank you, Sambit Panda

2 1

Removal of fortran FFTPACK
by Peter Bell July 25, 2019

July 25, 2019

Hello all, As of gh-10493<https://github.com/scipy/scipy/pull/10493>, pypocketfft now implements the discrete sine and cosine transforms meaning it is a complete replacement for the fortran FFTPACK library. In gh-10507<https://github.com/scipy/scipy/pull/10507>, I have proposed removing the fortran code from scipy completely and having scipy.fftpack depend on scipy.fft. This means that scipy.fftpack will now see some of the additional benefits of scipy.fft<https://github.com/scipy/scipy/wiki/Release-note-entries-for-SciPy-1.4.0#sc…> such as: long double transforms, GIL release, bluestein's algorithm, improved accuracy and improved multi-dimensional transform performance. However, there may be some cases where this affects users. For example, it will break anyone importing the private scipy.fftpack._fftpack submodule and the DCT and DST type-I are now about two times slower (but orders of magnitude more accurate). Regards, Peter

3 2

SciPy 1.0 preprint is up
by Tyler Reddy July 25, 2019

July 25, 2019

The community may like to know that the arXiv preprint of the SciPy 1.0 paper is up: https://arxiv.org/abs/1907.10121 The email with password to claim ownership on the paper should start circulating to co-authors soon. Submitting the manuscript to a peer reviewed journal is also likely to happen soon (again). Best wishes, Tyler

2 1

Re: [SciPy-Dev] Season of Docs applications
by Ralf Gommers July 25, 2019

July 25, 2019

Hi all, Here is a short update on Season of Docs. We are very happy and impressed with the number and quality of tech writers who applied to NumPy and SciPy. We decided we would really like to accept more people than the number of slots Google is likely to give us. We have found some more funding and mentoring bandwidth to be able to do so. After Google announces which candidates have been accepted, we will reach out to the next people on our shortlist and offer them the same mentoring and funding for their proposed projects as the accepted GSoD writers. I'm really looking forward to an interesting and productive fall working with several of you. Stay tuned! Cheers, Ralf On Wed, Jul 17, 2019 at 5:12 PM Ralf Gommers <ralf.gommers(a)gmail.com> wrote: > Hi GSoD applicants, > > To start with, I want to thank all of you who applied! There's a lot of > interest in helping improve NumPy's documentation and web presence, which > is awesome to see. During the application phase, we have interacted with > some of you already, and I will reach out to others over the next few days. > I'm just writing this email to say thank you and to confirm that yes we did > receive your application. Due to the volume of applications, it will be > difficult to reach out personally to all of you (I'll do my best though!), > hence this email. > > Really looking forward to what will happen next! > > Cheers, > Ralf > >

1 0

Combinatorial optimization in scipy.optimize
by Søren Fuglede Jørgensen July 22, 2019

July 22, 2019

Hi SciPy-Dev This question is motivated by the recent improvements to scipy.optimize.linear_sum_assignment. [0] Here, we get a much improved algorithm for solving minimum weight perfect matching in bipartite graphs (which is excellent). Now, this problem is one in a large family of combinatorial optimization problems on packing and covering, and what I'm wondering is if SciPy should be the home for a general class of efficient algorithms for solving those? Most of these can be phrased in terms of graph theory, and indeed networkx already contains implementations of many classical algorithms (and I was surprised to find that it didn't have anything matching linear_sum_assignment directly; it does have an implementation of the Blossom algorithm). So there's some overlap of scope here, but I guess that's not a problem in and by itself, but it had me wondering where the split should be? Just to phrase this in terms of (a very incomplete list of) examples (some of which have polynomial solutions, some of which don't), would there be appetite for including into scipy.optimize algorithms for solving e.g. 1. other covering/packing problems (e.g. min set/vertex/edge cover, maximum matching, ...), 2. maximum flow, 3. shortest path, 4. travelling salesman, 5. circuit minimization? Søren [0]: https://github.com/scipy/scipy/pull/10296

5 6

to our Season of Docs applicants
by Ralf Gommers July 18, 2019

July 18, 2019

Hi everyone, To start with, I want to thank all of you who applied for GSoD! There's a lot of interest in helping improve SciPy's documentation and web presence, which is awesome to see. During the application phase, we have interacted with some of you already, and I will reach out to others over the next few days. I'm just writing this email to say thank you and to confirm that yes we did receive your application. Due to the volume of applications, it will be difficult to reach out personally to all of you (I'll do my best though!), hence this email. Really looking forward to what will happen next! Cheers, Ralf

1 0

welcome Peter and Greg to the core team
by Ralf Gommers July 17, 2019

July 17, 2019

Hi all, On behalf of the SciPy developers I'd like to welcome Peter Bell and Gregory Lee as members of the core team. Peter is our current GSoC participant, and he's doing a really amazing job on scipy.fft. Also, he's commenting on other PRs/issues and pitching in in other places. Here is an overview of his SciPy PRs: https://github.com/scipy/scipy/pulls/peterbell10 Greg is one of the mentors on Peter's GSoC project (together with Eric Larson), has contributed to SciPy regularly in the past, and is also a maintainer of both PyWavelets and PyFFTW. Here is an overview of his SciPy PRs: https://github.com/scipy/scipy/pulls/grlee77 I'm looking forward to Peter and Greg's continued contributions! Cheers, Ralf

5 4

macOS minimum version and scipy.fft
by Peter Bell July 13, 2019

July 13, 2019

Hello All, Does anyone know what the minimum macOS version is that SciPy currently targets? And are there any objections to raising this to 10.9? The context for this is that in gh-10383<https://github.com/scipy/scipy/pull/10383> I am using uarray<https://uarray.readthedocs.io/en/latest/index.html> to add backend support to scipy.fft. uarray was recently rewritten in C++ and uses the thread_local keyword to support context-local backends. It's this single language feature that limits the version as older AppleClang versions didn't implement it. It can be worked around if support is an issue though. - Peter

2 1

Adding Poisson Mean Test to SciPy
by Mahendri Dwicahyo July 10, 2019

July 10, 2019

Hello scipy-dev community, I am Mahendri, an undergraduate student majoring in Computational Statistics. I notice that as of now, SciPy does not have two sample mean difference test from poisson distribution data. So I search for poisson mean test and I am interested in E-test written by Krishnamoorthy and Thomson because it has more power than its predecessor (C-test) and provide the code. The paper also quite popular with 126 citations. The code is written in Fortran but I am able to port that into python. The python implementation have dependencies for numpy and scipy.stats.poisson, the loop is removed and odd behavior found at original code is solved. Do you think it is possible to incorporate the test into SciPy? Thank you -- Mahendri Dwicahyo Computational Statistics, Polytechnic of Statistics STIS-BPS 15.8727(a)stis.ac.id | mahendri.md(a)gmail.com

1 0

Enabling use of scipy.special from Numba
by Joshua Wilson July 8, 2019

July 8, 2019

Hey everyone, For a while I've been thinking about how we can improve the compatibility of `scipy.special` with Numba. There is a way to do that currently by using `cython_special`: https://github.com/numba/numba/issues/3086 but that has a few drawbacks: (1) it's not very user friendly (2) it relies on the (I think?) private implementation details of Cython's `__pyx_capi__` (including the name mangling scheme used for fused functions) (3) it's clumsy for complex functions I think (3) requires changes on the Numba side, e.g. https://github.com/numba/numba/issues/3860 For (1) you could imagine us implementing another Numba specific API, but that runs into some immediate issues: - We'd need Numba as a dependency (which has been discussed: https://mail.python.org/pipermail/scipy-dev/2018-March/022576.html) - We don't want to just keep implementing new APIs For those reasons I'd prefer to find another solution. In an ideal world I'd like to solve (2) in such a way that it's easy for a third-party library to wrap `cython_special` in Numba jitted functions. There is a solution to that already that doesn't involve `__pyx_capi__`: a third-party library writes some Cython glue that exports PyCapsules of function pointers to the specializations of all the `cython_special` functions and then you grab those using ctypes to construct the jitted functions. For the Cython side of that see the docs on specialization: http://docs.cython.org/en/latest/src/userguide/fusedtypes.html#selecting-sp… But that feels rather convoluted. In an ideal world it would be nice to have an official way to grab the PyCapsules out of `__pyx_capi__` instead of writing a new set of capsules yourself. Note that we run into a similar problem with our own `LowLevelCallable`: https://docs.scipy.org/doc/scipy/reference/generated/scipy.LowLevelCallable… Getting the required name of the exported function for a fused function requires searching through the mangled names for the one you want. Out of all that, my main questions are: - Maybe I'm missing something obvious? Do people have better ideas for improving compatibility? - If not, would it be crazy to talk to the Cython folks about an official method for getting the PyCapsules out of `__pyx_capi__`? - Josh

6 16

scipy.fft submodule
by Peter Bell July 7, 2019

July 7, 2019

Following on from some of the discussion surrounding my backend proposal and the discussion on gh-10175<https://github.com/scipy/scipy/issues/10175>; I've been working on adding a new scipy.fft submodule. This is now working, tested, documented and ready for review in gh-10238<https://github.com/scipy/scipy/pull/10238>. The new submodule is almost a drop-in replacement for numpy.fft and scipy.fftpack, with a few exceptions: * Does not include NumPy's Hermitian transforms (hfft, ihfft) (but would be simple to add) * Uses numpy's conventions for rfft (complex array) instead of fftpack's (complex values packed into a real array) * Convolutions and pseudo-differential operators from fftpack are not included The new submodule adds pocketfft to implement the normal FFTs (not yet DCT and DST) which adds several new features beyond scipy.fftpack: * long double transforms * multi-dimensional real FFTs (rfftn) * Orthonormal transforms (norm='ortho' argument) * Bluestein's algorithm to avoid the worst case O(n^2) complexity of FFTPACK A few implementation details worth noting: * Unlike NumPy's version of pocketfft, this is C++ and uses templates to supporting the different floating point types * pocketfft also uses pybind11 which both adds a new build-time dependency and requires C++11. I believe this is the first use of C++11 in SciPy. * pocketfft has optional support for multithreading using OpenMP but this is currently disabled (and not compiled in at all). And finally, there is the rather awkward issue that scipy.fft already exists and is an alias for the numpy.fft.fft function. Currently I've had to add a workaround to allow the module scipy.fft to be callable, as discussed in gh-10253<https://github.com/scipy/scipy/issues/10253>. This is only relevant to code that imports scipy.fft so I don't expect it to be used often, if at all. Hopefully this is a temporary solution and the NumPy functions can be removed from the scipy namespace in some future release, or at the very least the FFT functions can be removed. Peter

4 5

Sprint at SciPy Conference?
by Matt Haberland July 4, 2019

July 4, 2019

The SciPy Conference <https://www.scipy2019.scipy.org/> approaches! Who would be interested in a sprint <https://www.scipy2019.scipy.org/sprints>? Can you participate in-person or virtually? We don't necessarily need a theme, but would you like to recruit help for something in particular? If you already registered a sprint on behalf of the SciPy library, please let me know. Otherwise, I can register (and I'll ask to be located near NumPy so people don't have to choose one or the other). Matt

3 5

ANN: SfePy 2019.2
by Robert Cimrman July 2, 2019

July 2, 2019

I am pleased to announce release 2019.2 of SfePy. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (limited support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: https://mail.python.org/mm3/mailman3/lists/sfepy.python.org/ Git (source) repository, issue tracker: https://github.com/sfepy/sfepy Highlights of this release -------------------------- - improved support for time-dependent homogenization problems, - Python 3.7 compatibility For full release notes see [1]. Cheers, Robert Cimrman [1] http://docs.sfepy.org/doc/release_notes.html#id1 --- Contributors to this release in alphabetical order: Robert Cimrman Jan Heczko Lubos Kejzlar Vladimir Lukes

1 0