Mailman 3 November 2022 - SciPy-Dev

Proposed Release Schedule for 1.10.0
by Tyler Reddy Nov. 7, 2022

Nov. 7, 2022

Hi, Let me know if you have any comments on the proposed release schedule below--it effectively mirrors the release cadence from last December. 1.9.x did a good job of testing the waters with meson and more recently the cibuildwheel infrastructure, so hopefully smooth sailing this time. - December 6/2022: branch 1.10.x - December 9/2022: rc1 - December 20/2022: rc2 (if needed) - January 3/2023: final release There are 30 open PRs and 25 open issues with a 1.10.0 milestone. Obviously, if you see one you can help with, that's great! Tyler

2 1

nightly musllinux wheels
by Andrew Nelson Nov. 4, 2022

Nov. 4, 2022

Dear list, we've just added musllinux testing to CI in eventual anticipation of creating musllinux wheels, which have often been requested. The testing process revealed some issues that can now be tackled. Anyway, I propose to add musllinux wheels to the wheel build, with them then being available from where we store them on anaconda. These wheels will be unofficial, we don't have to release them, but they may be useful downstream. What do people think? -- _____________________________________ Dr. Andrew Nelson _____________________________________

2 1

Optimize Pert distribution for stats.fit method
by Julian CHAMBRIER Nov. 3, 2022

Nov. 3, 2022

Hello, I use stats.fit method with custom Pert distribution. It works very well on standard data (like normal, poisson, ..) But on heterogeneous data, which does not necessarily form a pert distribution, I find it very slow. Could you help me to optimize it ? Thank you for your help, Sincerly class pertm_gen(rv_continuous): """ Implementation of modified PERT distribution With this implementation, we can use the scipy.stats.fit method to fit a PERT distribution on a sample To build PERT distribution, we use mainly the scipy Beta distribution """ def _shape_info(self): """ Informations on PERT distribution parameters """ low = _ShapeInfo("low", False, (0, np.inf), (False, False)) peak = _ShapeInfo("peak", False, (0, np.inf), (False, False)) high = _ShapeInfo("high", False, (0, np.inf), (False, False)) lmb = _ShapeInfo("lmb", False, (0, np.inf), (False, False)) return [low, peak, high, lmb] def _shape(self, low, peak, high, lmb): """ Method to modify Beta a and b parameters for PERT distribution @params : - low : Minimum parameter of the PERT distribution - peak : Mode parameter of the PERT distribution - high : Maximim parameter of the PERT distribution - lmb : Shaper parameter of the PERT distribution @return : - alpha and beta parameters for PERT distribution """ s_alpha = 1 + lmb*(peak - low)/(high-low) s_beta = 1 + lmb*(high - peak)/(high-low) return [s_alpha, s_beta] def _argcheck(self, low, peak, high, lmb): """ Method to check arguments @params : - low : Minimum parameter of the PERT distribution - peak : Mode parameter of the PERT distribution - high : Maximim parameter of the PERT distribution - lmb : Shaper parameter of the PERT distribution @return : - True if arguments satisfy all conditions, False otherwise """ return (low < peak) & (peak < high) & (low >= 0) & (lmb > 0) def _pdf(self, x, low, peak, high, lmb): """ Return the probability distribution function on the data @params : - x : Numeric array of data - low : Minimum parameter of the PERT distribution - peak : Mode parameter of the PERT distribution - high : Maximim parameter of the PERT distribution - lmb : Shaper parameter of the PERT distribution @return : - Return the probability distribution function on the data """ s_alpha, s_beta = self._shape(low, peak, high, lmb) x = ((x - low) / (high-low)) print(x) return beta_dist.pdf(x, s_alpha, s_beta) / (high-low) def _logpdf(self, x, low, peak, high, lmb): """ Return the log probability distribution function on the data @params : - x : Numeric array of data - low : Minimum parameter of the PERT distribution - peak : Mode parameter of the PERT distribution - high : Maximim parameter of the PERT distribution - lmb : Shaper parameter of the PERT distribution @return : - Return the log probability distribution function on the data """ return np.log(self._pdf(x, low, peak, high, lmb)) def _cdf(self, x, low, peak, high, lmb): """ Return the cumulative distribution function on the data @params : - x : Numeric array of data - low : Minimum parameter of the PERT distribution - peak : Mode parameter of the PERT distribution - high : Maximim parameter of the PERT distribution - lmb : Shaper parameter of the PERT distribution @return : - Return the cumulative distribution function on the data """ s_alpha, s_beta = self._shape(low, peak, high, lmb) x = ((x - low) / (high-low)) return beta_dist.cdf(x, s_alpha, s_beta) def _logcdf(self, x, low, peak, high, lmb): """ Return the log cumulative distribution function on the data @params : - x : Numeric array of data - low : Minimum parameter of the PERT distribution - peak : Mode parameter of the PERT distribution - high : Maximim parameter of the PERT distribution - lmb : Shaper parameter of the PERT distribution @return : - Return the log cumulative distribution function on the data """ return np.log(self._cdf(x, low, peak, high, lmb)) def _ppf(self, x, low, peak, high, lmb): """ Return the percent point function (inverse of cdf function) @params : - x : Numeric array of data - low : Minimum parameter of the PERT distribution - peak : Mode parameter of the PERT distribution - high : Maximim parameter of the PERT distribution - lmb : Shaper parameter of the PERT distribution @return : - Return the percent point function (inverse of cdf function) """ s_alpha, s_beta = self._shape(low, peak, high, lmb) return beta_dist.ppf(x, s_alpha, s_beta) * (high-low) + low def _rvs(self, low, peak, high, lmb, size=1, random_state=None): """ Returns a random sample from PERT distribution @params : - low : Minimum parameter of the PERT distribution - peak : Mode parameter of the PERT distribution - high : Maximim parameter of the PERT distribution - lmb : Shaper parameter of the PERT distribution - size : Indicate how many ra,dom values should be returned - random_state : Seed value for random sample RNG. @return : - Returns a random sample from PERT distribution """ s_alpha, s_beta = self._shape(low, peak, high, lmb) rvs_vals = (beta_dist(s_alpha, s_beta).rvs(size=size, random_state=random_state) * (high-low)) + low return rvs_vals pert = pertm_gen(name='pert')

2 2

Regarding fixing issue #17312 -- Fixing the docs
by Vinayak Dev Nov. 2, 2022

Nov. 2, 2022

Hey Everyone! I am a newbie developer looking forward to contribute to SciPy. I was searching for goo first issues when I came across issue #17312 which requests fixing the docstrings for scipy.ndImage showing that labels are 0-indexed,i.e , label l corresponds to l-1. I have already cloned the repo but cannot decide on the name for checkout branch and how to proceed further. Could anyone provide guidance? I would be very grateful. Thanks everyone! Vinayak Dev 'vinayakdsci' on github

2 3

CI flag for documentation PRs
by Daniel Schmitz Nov. 2, 2022

Nov. 2, 2022

Hi everyone, currently it is possible to skip running specific provider CI pipelines by for example adding `[skip cirrus]` to a commit message. For PRs which only update docstrings, these are still not specific enough and result in a lot of wasted CI resources. For example, building scipy on all platforms in the Azure pipelines would still be necessary as Azure pipelines include the refguide and lint check. Instead, I propose to add a CI flag like `[CI doc]` which would only execute the necessary checks for documentation updates. These would include: - refguide check - build and render docs - lint From what I understand, CircleCI also tests if scipy builds correctly (at least on one platform). In a PR I am currently working on, a copy-pasted non-unicode character in a docstring caused the build to fail during cythonization. This would then still be checked by CircleCI. I would appreciate any feedback about this proposal. Best

3 2

gh-17290: multi dimensional wasserstein/earth mover distance in Scipy
by Zehao Lu Nov. 1, 2022

Nov. 1, 2022

Hi all, I created issue #17290 (https://github.com/scipy/scipy/issues/17290) following up with #10382 (https://github.com/scipy/scipy/issues/10382) proposing to add a more general wasserstein (https://en.wikipedia.org/wiki/Wasserstein_metric)/earth mover (https://en.wikipedia.org/wiki/Earth_mover%27s_distance)/optimal transport distance in Scipy(a good material of the optimal transport theory is the book Computational Optimal Transport (https://arxiv.org/abs/1803.00567)) . For now, Scipy has its own scipy.stats.wasserstein_distance but it only works on one dimensional array. However, the wasserstien distance is naturally defined on any metric space and can be very useful in higher dimensional cases, i.e. color histogram comparison in computer vision area, or the document distance in NLP. In one-dimensional euclidean space, the wasserstein distance desirably equals the cdf distance(defined as the area between the two marginal cumulative distribution functions) but in other cases this does not hold, therefore some other backend methods for computing the higher dimensional wasserstein distance are needed. A naive approach is to solve the equivalent linear programming problem using the scipy.optimize.linprog method, it works but is comparably slow. To optimize LP many algorithms like network simplex can be used. Another method is the Sinkhorn algorithm(detailed explain can be found in Chapter 4 of Computational Optimal Transport (https://arxiv.org/abs/1803.00567)), which solves the optimal transport problem by adding a entropic penalty on the original cost function and applying the Lagrange multiplier method. I already wrote some scripts for the naive LP and the Sinkhorn algorithm, which can be found here -> https://github.com/com3dian/wass_nd/tree/main . I'm very willing to implement wasserstein_distance_nd and start a PR. Any ideas on this are welcomed in gh-17290 or by directly replying to this email. regards, Zehao Lu 'com3dian' on github

1 0

DOC: missing-bits: document recommendations on return object type
by Matt Haberland Nov. 1, 2022

Nov. 1, 2022

Hi team, gh-17310 proposes to add a note to the "Missing Bits" section of the developer documentation about the following recommendation: when a new function needs to return more than one conceptually distinct piece of information, a non-iterable class is preferable to a tuple, namedtuple, or bunch. This forces callers to be more explicit about the element of the returned object that they wish to access, and it makes it easier to extend the function or method in a backward-compatible way. Feedback is welcome at: https://github.com/scipy/scipy/pull/17310 Thanks! -- Matt Haberland Assistant Professor BioResource and Agricultural Engineering 08A-3K, Cal Poly

6 8