Mailman 3 February 2021 - SciPy-Dev

Speed up large NumPy arrays with PNumPy
by Matti Picus Feb. 18, 2021

Feb. 18, 2021

1 0

ANN: SciPy 1.6.1
by Tyler Reddy Feb. 18, 2021

Feb. 18, 2021

Hi all, On behalf of the SciPy development team I'm pleased to announce the release of SciPy 1.6.1, which is a bug fix release. Sources and binary wheels can be found at: https://pypi.org/project/scipy/ and at: https://github.com/scipy/scipy/releases/tag/v1.6.1 One of a few ways to install this release with pip: pip install scipy==1.6.1 ===================== SciPy 1.6.1 Release Notes ===================== SciPy 1.6.1 is a bug-fix release with no new features compared to 1.6.0. Please note that for SciPy wheels to correctly install with Pip on macOS 11, Pip >= 20.3.3 is needed. Authors ====== * Peter Bell * Evgeni Burovski * CJ Carey * Ralf Gommers * Peter Mahler Larsen * Cheng H. Lee + * Cong Ma * Nicholas McKibben * Nikola Forró * Tyler Reddy * Warren Weckesser A total of 11 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 1.6.1 ------------------------------- * `#13072 <https://github.com/scipy/scipy/issues/13072>`__: BLD: Quadpack undefined references * `#13241 <https://github.com/scipy/scipy/issues/13241>`__: Not enough values to unpack when passing tuple to \`blocksize\`... * `#13329 <https://github.com/scipy/scipy/issues/13329>`__: Large sparse matrices of big integers lose information * `#13342 <https://github.com/scipy/scipy/issues/13342>`__: fftn crashes if shape arguments are supplied as ndarrays * `#13356 <https://github.com/scipy/scipy/issues/13356>`__: LSQBivariateSpline segmentation fault when quitting the Python... * `#13358 <https://github.com/scipy/scipy/issues/13358>`__: scipy.spatial.transform.Rotation object can not be deepcopied... * `#13408 <https://github.com/scipy/scipy/issues/13408>`__: Type of \`has_sorted_indices\` property * `#13412 <https://github.com/scipy/scipy/issues/13412>`__: Sorting spherical Voronoi vertices leads to crash in area calculation * `#13421 <https://github.com/scipy/scipy/issues/13421>`__: linear_sum_assignment - support for matrices with more than 2^31... * `#13428 <https://github.com/scipy/scipy/issues/13428>`__: \`stats.exponnorm.cdf\` returns \`nan\` for small values of \`K\`... * `#13465 <https://github.com/scipy/scipy/issues/13465>`__: KDTree.count_neighbors : 0xC0000005 error for tuple of different... * `#13468 <https://github.com/scipy/scipy/issues/13468>`__: directed_hausdorff issue with shuffle * `#13472 <https://github.com/scipy/scipy/issues/13472>`__: Failures on FutureWarnings with numpy 1.20.0 for lfilter, sosfilt... * `#13565 <https://github.com/scipy/scipy/issues/13565>`__: BUG: 32-bit wheels repo test failure in optimize Pull requests for 1.6.1 ----------------------------- * `#13318 <https://github.com/scipy/scipy/pull/13318>`__: REL: prepare for SciPy 1.6.1 * `#13344 <https://github.com/scipy/scipy/pull/13344>`__: BUG: fftpack doesn't work with ndarray shape argument * `#13345 <https://github.com/scipy/scipy/pull/13345>`__: MAINT: Replace scipy.take with numpy.take in FFT function docstrings. * `#13354 <https://github.com/scipy/scipy/pull/13354>`__: BUG: optimize: rename private functions to include leading underscore * `#13387 <https://github.com/scipy/scipy/pull/13387>`__: BUG: Support big-endian platforms and big-endian WAVs * `#13394 <https://github.com/scipy/scipy/pull/13394>`__: BUG: Fix Python crash by allocating larger array in LSQBivariateSpline * `#13400 <https://github.com/scipy/scipy/pull/13400>`__: BUG: sparse: Better validation for BSR ctor * `#13403 <https://github.com/scipy/scipy/pull/13403>`__: BUG: sparse: Propagate dtype through CSR/CSC constructors * `#13414 <https://github.com/scipy/scipy/pull/13414>`__: BUG: maintain dtype of SphericalVoronoi regions * `#13422 <https://github.com/scipy/scipy/pull/13422>`__: FIX: optimize: use npy_intp to store array dims for lsap * `#13425 <https://github.com/scipy/scipy/pull/13425>`__: BUG: spatial: make Rotation picklable * `#13426 <https://github.com/scipy/scipy/pull/13426>`__: BUG: \`has_sorted_indices\` and \`has_canonical_format\` should... * `#13430 <https://github.com/scipy/scipy/pull/13430>`__: BUG: stats: Fix exponnorm.cdf and exponnorm.sf for small K * `#13470 <https://github.com/scipy/scipy/pull/13470>`__: MAINT: silence warning generated by \`spatial.directed_hausdorff\` * `#13473 <https://github.com/scipy/scipy/pull/13473>`__: TST: fix failures due to new FutureWarnings in NumPy 1.21.dev0 * `#13479 <https://github.com/scipy/scipy/pull/13479>`__: MAINT: update directed_hausdorff Cython code * `#13485 <https://github.com/scipy/scipy/pull/13485>`__: BUG: KDTree weighted count_neighbors doesn't work between two... * `#13503 <https://github.com/scipy/scipy/pull/13503>`__: TST: fix \`test_fortranfile_read_mixed_record\` on big-endian... * `#13518 <https://github.com/scipy/scipy/pull/13518>`__: DOC: document that pip >= 20.3.3 is needed for macOS 11 * `#13520 <https://github.com/scipy/scipy/pull/13520>`__: BLD: update reqs based on oldest-supported-numpy in pyproject.toml * `#13567 <https://github.com/scipy/scipy/pull/13567>`__: TST, BUG: adjust tol on test_equivalence Checksums ========= MD5 ~~~ 6312f6644420a0ad11f9dfb80aaa0560 scipy-1.6.1-cp37-cp37m-macosx_10_9_x86_64.whl 0018622e5d32ca0cc690db152a371889 scipy-1.6.1-cp37-cp37m-manylinux1_i686.whl 7612dc5ebc5928d606b6f486e0edabad scipy-1.6.1-cp37-cp37m-manylinux1_x86_64.whl bcbc57efab027e9c74fe4be8ac1b6470 scipy-1.6.1-cp37-cp37m-manylinux2014_aarch64.whl 49d9b5824b22c87d184214497fec1079 scipy-1.6.1-cp37-cp37m-win32.whl 929834c270b3056997717bbcee58809c scipy-1.6.1-cp37-cp37m-win_amd64.whl 4a862104bb2add633ead9a28496356ae scipy-1.6.1-cp38-cp38-macosx_10_9_x86_64.whl c0dc4f798d0acc015c5fb36d3d97f4ed scipy-1.6.1-cp38-cp38-manylinux1_i686.whl 8f0dce3503871db857f44a3ffb5800f6 scipy-1.6.1-cp38-cp38-manylinux1_x86_64.whl e4ee2176f25684d1cd3d21f0db5906ed scipy-1.6.1-cp38-cp38-manylinux2014_aarch64.whl 8589661ea9a320746aef8299cd16f32f scipy-1.6.1-cp38-cp38-win32.whl 819424a909991eec489441880709a97c scipy-1.6.1-cp38-cp38-win_amd64.whl e7ea30f4dc26b79a3a2b9446afd4c572 scipy-1.6.1-cp39-cp39-macosx_10_9_x86_64.whl d8f7678b426174aba4a6184803d90c5a scipy-1.6.1-cp39-cp39-manylinux1_i686.whl d8f5ec24b15fef9786a6233c28003753 scipy-1.6.1-cp39-cp39-manylinux1_x86_64.whl 4a832944f71c5f7b019f6539475647a2 scipy-1.6.1-cp39-cp39-manylinux2014_aarch64.whl 5fff9d3f673e4ae73e76f02ea8544aa3 scipy-1.6.1-cp39-cp39-win32.whl b03f9713b7b9be7fa019ab3c94c72254 scipy-1.6.1-cp39-cp39-win_amd64.whl 98a860ce2d6296cace333d0a07501f13 scipy-1.6.1.tar.gz 5cd15c4b4abf2e24ed05dbde9e7b90c8 scipy-1.6.1.tar.xz a3c4bf7491ea0ab49bc8b149334f50f7 scipy-1.6.1.zip SHA256 ~~~~~~ a15a1f3fc0abff33e792d6049161b7795909b40b97c6cc2934ed54384017ab76 scipy-1.6.1-cp37-cp37m-macosx_10_9_x86_64.whl e79570979ccdc3d165456dd62041d9556fb9733b86b4b6d818af7a0afc15f092 scipy-1.6.1-cp37-cp37m-manylinux1_i686.whl a423533c55fec61456dedee7b6ee7dce0bb6bfa395424ea374d25afa262be261 scipy-1.6.1-cp37-cp37m-manylinux1_x86_64.whl 33d6b7df40d197bdd3049d64e8e680227151673465e5d85723b3b8f6b15a6ced scipy-1.6.1-cp37-cp37m-manylinux2014_aarch64.whl 6725e3fbb47da428794f243864f2297462e9ee448297c93ed1dcbc44335feb78 scipy-1.6.1-cp37-cp37m-win32.whl 5fa9c6530b1661f1370bcd332a1e62ca7881785cc0f80c0d559b636567fab63c scipy-1.6.1-cp37-cp37m-win_amd64.whl bd50daf727f7c195e26f27467c85ce653d41df4358a25b32434a50d8870fc519 scipy-1.6.1-cp38-cp38-macosx_10_9_x86_64.whl f46dd15335e8a320b0fb4685f58b7471702234cba8bb3442b69a3e1dc329c345 scipy-1.6.1-cp38-cp38-manylinux1_i686.whl 0e5b0ccf63155d90da576edd2768b66fb276446c371b73841e3503be1d63fb5d scipy-1.6.1-cp38-cp38-manylinux1_x86_64.whl 2481efbb3740977e3c831edfd0bd9867be26387cacf24eb5e366a6a374d3d00d scipy-1.6.1-cp38-cp38-manylinux2014_aarch64.whl 68cb4c424112cd4be886b4d979c5497fba190714085f46b8ae67a5e4416c32b4 scipy-1.6.1-cp38-cp38-win32.whl 5f331eeed0297232d2e6eea51b54e8278ed8bb10b099f69c44e2558c090d06bf scipy-1.6.1-cp38-cp38-win_amd64.whl 0c8a51d33556bf70367452d4d601d1742c0e806cd0194785914daf19775f0e67 scipy-1.6.1-cp39-cp39-macosx_10_9_x86_64.whl 83bf7c16245c15bc58ee76c5418e46ea1811edcc2e2b03041b804e46084ab627 scipy-1.6.1-cp39-cp39-manylinux1_i686.whl 794e768cc5f779736593046c9714e0f3a5940bc6dcc1dba885ad64cbfb28e9f0 scipy-1.6.1-cp39-cp39-manylinux1_x86_64.whl 5da5471aed911fe7e52b86bf9ea32fb55ae93e2f0fac66c32e58897cfb02fa07 scipy-1.6.1-cp39-cp39-manylinux2014_aarch64.whl 8e403a337749ed40af60e537cc4d4c03febddcc56cd26e774c9b1b600a70d3e4 scipy-1.6.1-cp39-cp39-win32.whl a5193a098ae9f29af283dcf0041f762601faf2e595c0db1da929875b7570353f scipy-1.6.1-cp39-cp39-win_amd64.whl c4fceb864890b6168e79b0e714c585dbe2fd4222768ee90bc1aa0f8218691b11 scipy-1.6.1.tar.gz 2800f47a5040cbab05b3ce58f1dfb670c70232b0f56d30380c6fd4ef4e787df5 scipy-1.6.1.tar.xz 18601effa06aba0e9f34475b6d34b3d7454feabe8b0f96bcc483b3fd38b0afc2 scipy-1.6.1.zip

1 0

Implementation of the Alexander-Govern test
by Warren Weckesser Feb. 17, 2021

Feb. 17, 2021

Hey all, The pull request https://github.com/scipy/scipy/pull/12873 to add the Alexander-Govern test (https://www.jstor.org/stable/1165140), a test currently in the detailed roadmap for the stats module, looks ready to merge. I'd like to merge on Friday, so be sure to comment in the PR if you have any issues with the implementation. Thanks! Warren

1 0

Re: [SciPy-Dev] Pull request test cancelled
by Ramón Casero Cañas Feb. 16, 2021

Feb. 16, 2021

Good evening, The scipy.scipy test timed out again after 60 min for https://github.com/scipy/scipy/pull/13566. Could somebody restart it, please? Kind regards, Ramon. On Mon, 15 Feb 2021 at 22:00, Ramón Casero Cañas <rcasero(a)gmail.com> wrote: > Hi, > > i submitted a pull request to speed up hdquantile_sd ( > https://github.com/scipy/scipy/pull/13566). > > I think all tests passed, except one that was cancelled, but I don't think > that has to do with my changes? > > azure-pipelines/ scipy.scipy > > Build log #L650 > > The operation was canceled. > > > Kind regards, > > Ramon. > > -- > Dr Ramón Casero, CSci, Post-Doctoral Researcher > > > MRC Harwell Institute, Biocomputing, Mammalian Genetics Unit > Harwell Campus, Oxfordshire, OX11 0RD, UK. > Tel: +44 (0)1235 841237 | www.har.mrc.ac.uk > -- Dr Ramón Casero, CSci, Post-Doctoral Researcher MRC Harwell Institute, Biocomputing, Mammalian Genetics Unit Harwell Campus, Oxfordshire, OX11 0RD, UK. Tel: +44 (0)1235 841237 | www.har.mrc.ac.uk

1 1

New probability distributions in stats: the noncentral hypergeometric distributions
by Warren Weckesser Feb. 16, 2021

Feb. 16, 2021

Hey all, Matt Haberland's pull request with implementations of the Fisher and Wallenius noncentral hypergeometric distributions (built on Nicholas McKibben's wrappers of Agner Fog's biasedurn code) looks ready to merge: https://github.com/scipy/scipy/pull/13330. It would be nice to get some more eyes on it. The last issue that we've been discussing is the names of the distributions. The current names, `fnch` and `wnch`, are quite concise, and some longer alternatives are being considered in the PR. Comments on that and on the PR in general would be helpful. Thanks! Warren

1 0

Pull request test cancelled
by Ramón Casero Cañas Feb. 15, 2021

Feb. 15, 2021

Hi, i submitted a pull request to speed up hdquantile_sd ( https://github.com/scipy/scipy/pull/13566). I think all tests passed, except one that was cancelled, but I don't think that has to do with my changes? azure-pipelines/ scipy.scipy Build log #L650 The operation was canceled. Kind regards, Ramon. -- Dr Ramón Casero, CSci, Post-Doctoral Researcher MRC Harwell Institute, Biocomputing, Mammalian Genetics Unit Harwell Campus, Oxfordshire, OX11 0RD, UK. Tel: +44 (0)1235 841237 | www.har.mrc.ac.uk

2 2

Boost for stats
by Nicholas McKibben Feb. 15, 2021

Feb. 15, 2021

Hi all, Many stats distributions in SciPy have outstanding issues with difficult solutions in legacy code. We've been working on replacing existing statistical distributions with those found in Boost.Math. The initial implementation resolves almost a dozen issues for scipy.stats with potential for resolving several more in scipy.stats and scipy.special in future PRs. Initial PR: https://github.com/scipy/scipy/pull/1332 <https://github.com/scipy/scipy/pull/13328> This PR includes the ability to easily add Boost functionality through generated ufuncs. Boost is a large library and would incur the cost of one of the follow - an additional dependency (e,g, boostinator https://github.com/mckib2/boostinator) that outsources the packaging of the Boost libraries - the inclusion of Boost within SciPy either as a "clone and own" or submodule The initial PR includes the zipped Boost headers only (~24MB zipped), but adding Boost as a submodule might be a more maintainable approach if changes to Boost need to be made in the future. Inclusion of the entire Boost library is a virtual necessity for the Boost.Math module. Manual attempts to strip away unnecessary files and bcp (Boost's utility to provide stripped down installations) fail to create smaller sizes. The increase in size would be similar to the following: SciPy master repo ~177 MB Boost branch: ~221 MB Built: ~939 MB Built With Boost: ~1090 MB Wheel size should not be significantly impacted because Boost is used as a header-only library. I have no relationship with the Boost libraries other than as a user and bug reporter. I find them to be impressive and well-maintained with tremendous support from both industry and open source developers. SciPy would benefit from the efficient, well-tested and maintained implementations of stats and special algorithms. Thanks, Nicholas

9 12

Re: [SciPy-Dev] GSoC'21 participation SciPy
by Pamphile Roy Feb. 15, 2021

Feb. 15, 2021

> I'm assuming you mean something like standard multiprocessing, or using a > custom Pool object, for code that's trivially parallelizable. Both are > covered by the `workers` pattern. If you're thinking about something else, > can you elaborate? Yes the first step would be to do some simple multiprocessing. But IMO we should still try to have something flexible enough so that we could plug (or the user) something else like a job scheduler. I would not see any scheduling per say in SciPy (totally out of scope and too many options: bash, slurm, kubernetes world, etc.), but being able to access a queue would be nice. This way you could wrap anything to go take a job form a queue and return a result. This would need some experiments, but I think that we could achieve something like this just with some simple queue from std lib. Because I am not sure we would want to introduce, even optionally, a dependency to something like RabbitMQ. (Or maybe…). > Not sure about EGO in particular (I'm not familiar with it), gaussian > processes sounds a little out of scope - that's scikit-learn territory > probably. True. I am using scikit-optimize for that currently. It just seemed EGO was getting more and more tractions and so I though that maybe having this in SciPy could be justified. But definitely, other famous algorithms from the list you posted would be good candidates. > I'm not 100% sure, let's see if someone more familiar with this topic has > an opinion. In general for new stats functionality we try to figure out if > it fits better in scipy.stats or in statsmodels. The latter doesn't have > much either right now, only: > https://www.statsmodels.org/stable/generated/statsmodels.genmod.generalized… I posted several time the suggestion to statsmodels mailing list but so far did get a clear answer. Hopefully we will have more luck here :) Agreed for the response surface part. It would be a big undertaking. Cheers, Pamphile @tupui

1 0

GSoC'21 participation SciPy
by Ralf Gommers Feb. 14, 2021

Feb. 14, 2021

Hi all, It's GSoC time soon! I'd like to participate again under the PSF umbrella this year. The GSoC project durations have been halved, which means they're also more manageable to mentor. They're 175 hours now. For more details, see https://summerofcode.withgoogle.com/. Also worth mentioning is that NumPy decided not to participate, but a few people on the NumPy team are interested in helping mentor on a SciPy project. We've also been discussing an initiative around mentoring of people new to open source from under-represented backgrounds with the NumPy team, and came to the conclusion that it'd be better to use SciPy project ideas and help mentor them. So if you have ideas that don't exactly fit the size or timeline for GSoC, please still propose them since they may be suitable for that initiative. We have a few more weeks till the deadline to sign up, so it'd be great to get some ideas and potential mentors onto this ideas page: https://github.com/scipy/scipy/wiki/GSoC-2021-project-ideas. Any thoughts, good project ideas, or volunteers to mentor? Cheers, Ralf

1 0

Hyper-dual numbers
by Andrea Cortis Feb. 14, 2021

Feb. 14, 2021

Hello, first time here. I was wondering if there are plans for the definition of a “hyper-dual” type in numpy. I think that would be most useful for neural nets training, and optimization in general. Andrea Sent from my iPad

5 11