Mailman 3 August 2020 - SciPy-Dev

proposal to drop Python 3.6 and NumPy 1.14 / 1.15
by Ralf Gommers 15 Nov '20

15 Nov '20

Hi all, Python 3.9 is coming out before the next SciPy release, and NEP 29 recommends to support only Python >=3.7 and NumPy >= 1.16 from last month onwards [1]. I think supporting 3 Python versions and 5 NumPy versions in SciPy 1.6.0 is easily enough. That would also bring us back in line with SciPy on conda-forge, which built against NumPy 1.16 for 1.5.2. Any objections? Cheers, Ralf [1] https://numpy.org/neps/nep-0029-deprecation_policy.html#support-table

10 21

Adding non-parametric methods to scipy.stats
by Romain Jacob 27 Oct '20

27 Oct '20

Hello everyone, I have been working for some time on the implementation of non-parametric methods to compute confidence intervals for percentiles. There are some very interesting results in the literature (see e.g. a nice pitch in [1]) which I think it would be great to add to SciPy to make them more readily available. It also seems to be rather in line with "recent" discussions of the roadmap for scipy.stats [2]. I would be interested in contributing this. What do you think? Cheers, -- Romain [1] https://ieeexplore.ieee.org/document/6841797 [2] https://github.com/scipy/scipy/issues/10577 -- Romain Jacob Postdoctoral Researcher ETH Zurich - Computer Engineering and Networks Laboratory www.romainjacob.net <https://www.romainjacob.net/> @RJacobPartner <https://twitter.com/RJacobPartner> Gloriastrasse 35, ETZ G75 8092 Zurich +41 7 68 16 88 22

6 18

Re: [SciPy-Dev] Discontinued Rackspace Open Compute Discount
by Robert Lucente 05 Oct '20

05 Oct '20

Several CI services are free for open source projects although there are throughput limits https://about.gitlab.com https://github.com/features/actions https://travis-ci.org https://www.appveyor.com -----Original Message----- >From: Olivier Grisel <olivier.grisel(a)ensta.org> >Sent: Feb 8, 2020 11:30 AM >To: SciPy Developers List <scipy-dev(a)python.org> >Subject: [SciPy-Dev] Discontinued Rackspace Open Compute Discount > >Hi all, > >I received an email end of January that I just discovered today >telling me that the Rackspace Open Compute Discount grant we use to >host http://wheels.scipy.org for nightly builds and release management >of our community has been discontinued on December 31st 2019. > >I would like to take down the Rackspace hosted cloudfiles as soon as >possible but we need to find a replacement as this will break all the >CIs that rely on nightly wheels and also all the wheel release >infrastructure for several project in the ecosystem. > >Apparently the fees for January and the beginning of February already >sum to $936.51. This is a large cost because there were a bunch of >unused left-over block volumes that I already deleted. > >To host wheels for nightly builds and release automation, a possible >alternatives include using the github release pages but they are not >really designed for this kind of workflow. > >Alternatively there is a PyPI repo hosting feature in Azure Pipelines >in public preview but I am not sure how it works, if it's free for >open source projects and if we could use it as a replacement for the >Rackspace cloud storage we currently use to host >http://wheels.scipy.org . > >https://docs.microsoft.com/en-us/azure/devops/pipelines/artifacts/pypi?view… > >Important: if you have uploaded any important files to one of the >cloud file storage / blob containers on Rackspace, please download a >copy as soon as possible. > >But please do not download things you do not really want to keep to >avoid adding any unnecessary bandwidth costs. If you have file write >permission on one of those containers, please delete the files you own >and do not need anymore. > >Feel free to join this room to discuss the matter interactively: >https://gitter.im/scikit-learn/nightly-build-hosting > >Feel free to reply to this email on the scipy-dev mailing list if you >have suggestions for setting up a free replacement. > >-- >Olivier >http://twitter.com/ogrisel - http://github.com/ogrisel >_______________________________________________ >SciPy-Dev mailing list >SciPy-Dev(a)python.org >https://mail.python.org/mailman/listinfo/scipy-dev

12 25

Thanks wiki protocol
by rlucas7＠vt.edu 01 Sep '20

01 Sep '20

Hi scipy-dev, For work on the 1.5 release we used a wiki rather than have people add commits to the thanks file in the repo. I’m assuming this was to alleviate rebase conflicts to merge. My question is, Is using a wikifor tracking thanks to add in each release the way we track thanks entries moving forward? To be clear these would be separate wiki pages from the release notes wiki pages. -Lucas Roberts

4 8

Proposal for a new ODE Solver
by Laurin Steidle 30 Aug '20

30 Aug '20

Hi all, I opened an github issue [1] asking the following and was recommended to post it here. I am *proposing to add a new ODE solver* to the list of solvers available in solve_ivp. The solver I propose is the so called "*modified Patanka-Runge-Kutta" (MPRK)* [2] scheme. MPRK is a integration scheme that satisfies the condition of 'unconditional positivity', a requirement for many ODEs especially in the field of biochemical models. A simple and famous example of such a ODE being the Lotka-Volterra equations. So far there exists no such solver in scipy. Hence, a ODE solution with the present solvers can become "unphysical" and therefore invalid if the solution at a any time-step becomes negative due to numerical errors. If this is something the community agrees on adding I am happy to write the implementation. However, I would like to ask for assistance by a more experienced scipy dev as I am new to scipy. Cheers, Laurin [1] https://github.com/scipy/scipy/issues/12766 [2] https://doi.org/10.1016/S0168-9274(03)00101-6

2 1

More robust scalar root finding
by Juan Luis Cano 28 Aug '20

28 Aug '20

Hi all, A month ago I was trying to migrate some MATLAB code to Python and found a couple of annoyances in the scalar root finding functions of SciPy. I opened these issues: "Make root_scalar work when only x0 is given, similar to newton" https://github.com/scipy/scipy/issues/12620 "newton chokes when iteration steps outside function domain" https://github.com/scipy/scipy/issues/12625 and a pull request to address #12625: "Make scalar, iterative root finding methods robust against steps outside domain" https://github.com/scipy/scipy/pull/12626 It seems that the conversation has stalled, so I wanted to bring them to the mailing list. As I said in #12625, common wisdom dictates that if one chooses a starting guess closer to the real solution, convergence is achieved. However, leaving this well known numerical fact aside for a moment, I wonder if some work could be done on the *usability* side of things to (a) make root_scalar a drop-in replacement to newton (#12620) (b) have some opt-out way to increase the robustness of the solution in a way that "just works", which is that MATLAB and Octave achieve (#12625) A personal side note: The original authors of the code I'm migrating tried Python half a decade ago but noticed that "SciPy was not converging, while MATLAB had no problems", so they ended up using the latter. While we all scoffed when the infamous "top reasons to choose MATLAB" comparison was published [1][2][3][4], I do think there's still a lot to understand from a UX point of view of why people choose one or another. After 7 years of teaching Python to engineers, they either become data scientists or they go back to MATLAB, and as a result I have decided to get my hands dirty and find out the reasons, to the best of my possibilities. Best, Juan Luis [1] https://web.archive.org/web/20190806100605/https://www.mathworks.com/produc… [2] https://twitter.com/moorepants/status/1158248132355874817 [3] https://twitter.com/randal_olson/status/1158402855818018816 [4] https://twitter.com/neuromusic/status/1158556716096618497 [5] https://twitter.com/vboykis/status/1158559458173313024

3 3

Wallenius hypergeometric distribution
by Harry Gulliver 10 Aug '20

10 Aug '20

Hi all, new (potential) contributor here! I'm working on a project using Wallenius' non-central hypergeometric distribution atm and noticed it's not yet available in scipy.stats, so thought I'd volunteer to add it. Could possibly also do Fisher's hypergeom while I'm at it. Anything I ought to know before just writing some code and making a pull request? How can I best help? Thanks! Harry

3 4

Changing return types in a public API?
by Peter Bell 06 Aug '20

06 Aug '20

Hello all, I've been working towards unifying KDTree and cKDTree in gh-12382<https://github.com/scipy/scipy/pull/12382#discussion_r451491803>. This has raised some general questions about API compatibility that I'd appreciate any thoughts on. First, KDTree returns NumPy scalars everywhere because the results come from indexing NumPy arrays. Whereas, cKDTree returns python ints wherever possible. Is it reasonable for an API to change from returning NumPy scalars to python int? The NumPy scalars do mimic the array interface to some extent so there is a small interface incompatibility. However, the documentation usually just says a function returns int or float, not NumPy scalar specifically. Secondly, KDTree uses `dtype=int` everywhere which results in 64-bit integers on linux but 32-bit integers on windows. Ideally, I'd want to return 64-bit integers (or at least np.intp) on all platforms for consistency and to avoid issues with integer overflow. Again, this is behaviour that isn't documented but code could still be relying on implicitly. - Peter

4 3

Implementing fit method for discrete distributions
by Эмиль Лаккис 06 Aug '20

06 Aug '20

Hi everyone, I am a newcomer here and after reading through #11948 I thought implementing fit option for discrete distributions could be a good start. In particular, I was thinking of implementing MLE + MM (whenever possible). Do people generally think this is interesting? If so, is it generally expected to provide references for every estimator being used? Thank you! Best, Emil

1 0

ENH: Add an approximate Linear Sum Assignment Solver
by Ali Saad-Eldin 04 Aug '20

04 Aug '20

Hi all, Hi all, I would like to add an approximate Linear Assignment Problem<http://www.assignmentproblems.com/doc/LSAPIntroduction.pdf> (aLAP) solver to scipy.optimize. The specific approximation algorithm I propose adding can be found here<https://link.springer.com/content/pdf/10.1007%2F978-3-540-68111-3_74.pdf>. Scipy.optimize currently houses an exact solver for LAP in linear_sum_assignment()<https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linear_…>, and I was thinking this could be added either as a new method within linear_sum_assignment(), or as its own function. The approximation algorithm has time complexity O(n2log(n)), compared to the algorithm implemented in linear_sum_assignment() with complexity ~O(n2.3 ), while guaranteeing a score >= 1/2 optimum (though in practice the scores appear to be much closer to optimum). I've attached two plots demonstrating runtime and accuracy, running linear_sum_assignment() and aLAP on simulated, dense nxn cost_matrices with entries randomly selected from a uniform(0,100) distribution, for n sampled from [0, 3000]. Note that linear_sum_assignment() is essentially C/C++, while the aLAP implementation is native Python, so a cython implementation could make aLAP even faster. Additionally, an advantage to this algorithm is that it is parallelizable, which could cause a further speed up. Current version of the implementation can be found here<https://github.com/neurodata/graspy/blob/ase-alap/graspy/match/alap.py>, and proof of effectiveness here<https://nbviewer.jupyter.org/github/asaadeldin11/nbview/blob/master/aLAP_co…>. Best, Ali Saad-Eldin

2 1