SciPy 1.10.0 is the culmination of 6 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Before upgrading, we recommend that users check that their own code does not use deprecated SciPy functionality (to do so, run your code with ``python -Wd`` and check for ``DeprecationWarning`` s). Our development attention will now shift to bug-fix releases on the 1.10.x branch, and on adding new features on the main branch.
This release requires Python 3.8+ and NumPy 1.19.5 or greater.
For running on PyPy, PyPy3 6.0+ is required.
**************************** Highlights of this release ****************************
- A new dedicated datasets submodule (`scipy.datasets`) has been added, and is now preferred over usage of `scipy.misc` for dataset retrieval. - A new `scipy.interpolate.make_smoothing_spline` function was added. This function constructs a smoothing cubic spline from noisy data, using the generalized cross-validation (GCV) criterion to find the tradeoff between smoothness and proximity to data points. - `scipy.stats` has three new distributions, two new hypothesis tests, three new sample statistics, a class for greater control over calculations involving covariance matrices, and many other enhancements.
**************** New features ****************
`scipy.datasets` introduction ====================== - A new dedicated ``datasets`` submodule has been added. The submodules is meant for datasets that are relevant to other SciPy submodules ands content (tutorials, examples, tests), as well as contain a curated set of datasets that are of wider interest. As of this release, all the datasets from `scipy.misc` have been added to `scipy.datasets` (and deprecated in `scipy.misc`). - The submodule is based on [Pooch](https://www.fatiando.org/pooch/latest/) (a new optional dependency for SciPy), a Python package to simplify fetching data files. This move will, in a subsequent release, facilitate SciPy to trim down the sdist/wheel sizes, by decoupling the data files and moving them out of the SciPy repository, hosting them externally and downloading them when requested. After downloading the datasets once, the files are cached to avoid network dependence and repeated usage. - Added datasets from ``scipy.misc``: `scipy.datasets.face`, `scipy.datasets.ascent`, `scipy.datasets.electrocardiogram` - Added download and caching functionality:
- `scipy.datasets.download_all`: a function to download all the `scipy.datasets` associated files at once. - `scipy.datasets.clear_cache`: a simple utility function to clear cached dataset files from the file system. - ``scipy/datasets/_download_all.py`` can be run as a standalone script for packaging purposes to avoid any external dependency at build or test time. This can be used by SciPy packagers (e.g., for Linux distros) which may have to adhere to rules that forbid downloading sources from external repositories at package build time.
`scipy.integrate` improvements ======================== - Added `scipy.integrate.qmc_quad`, which performs quadrature using Quasi-Monte Carlo points. - Added parameter ``complex_func`` to `scipy.integrate.quad`, which can be set ``True`` to integrate a complex integrand.
`scipy.interpolate` improvements ========================= - `scipy.interpolate.interpn` now supports tensor-product interpolation methods (``slinear``, ``cubic``, ``quintic`` and ``pchip``) - Tensor-product interpolation methods (``slinear``, ``cubic``, ``quintic`` and ``pchip``) in `scipy.interpolate.interpn` and `scipy.interpolate.RegularGridInterpolator` now allow values with trailing dimensions. - `scipy.interpolate.RegularGridInterpolator` has a new fast path for ``method="linear"`` with 2D data, and ``RegularGridInterpolator`` is now easier to subclass - `scipy.interpolate.interp1d` now can take a single value for non-spline methods. - A new ``extrapolate`` argument is available to `scipy.interpolate.BSpline.design_matrix`, allowing extrapolation based on the first and last intervals. - A new function `scipy.interpolate.make_smoothing_spline` has been added. It is an implementation of the generalized cross-validation spline smoothing algorithm. The ``lam=None`` (default) mode of this function is a clean-room reimplementation of the classic ``gcvspl.f`` Fortran algorithm for constructing GCV splines. - A new ``method="pchip"`` mode was aded to `scipy.interpolate.RegularGridInterpolator`. This mode constructs an interpolator using tensor products of C1-continuous monotone splines (essentially, a `scipy.interpolate.PchipInterpolator` instance per dimension).
`scipy.sparse.linalg` improvements =========================== - The spectral 2-norm is now available in `scipy.sparse.linalg.norm`. - The performance of `scipy.sparse.linalg.norm` for the default case (Frobenius norm) has been improved. - LAPACK wrappers were added for ``trexc`` and ``trsen``. - The `scipy.sparse.linalg.lobpcg` algorithm was rewritten, yielding the following improvements:
- a simple tunable restart potentially increases the attainable accuracy for edge cases, - internal postprocessing runs one final exact Rayleigh-Ritz method giving more accurate and orthonormal eigenvectors, - output the computed iterate with the smallest max norm of the residual and drop the history of subsequent iterations, - remove the check for ``LinearOperator`` format input and thus allow a simple function handle of a callable object as an input, - better handling of common user errors with input data, rather than letting the algorithm fail.
`scipy.linalg` improvements ===================== - `scipy.linalg.lu_factor` now accepts rectangular arrays instead of being restricted to square arrays.
`scipy.ndimage` improvements ======================== - The new `scipy.ndimage.value_indices` function provides a time-efficient method to search for the locations of individual values with an array of image data. - A new ``radius`` argument is supported by `scipy.ndimage.gaussian_filter1d` and `scipy.ndimage.gaussian_filter` for adjusting the kernel size of the filter.
`scipy.optimize` improvements ======================== - `scipy.optimize.brute` now coerces non-iterable/single-value ``args`` into a tuple. - `scipy.optimize.least_squares` and `scipy.optimize.curve_fit` now accept `scipy.optimize.Bounds` for bounds constraints. - Added a tutorial for `scipy.optimize.milp`. - Improved the pretty-printing of `scipy.optimize.OptimizeResult` objects. - Additional options (``parallel``, ``threads``, ``mip_rel_gap``) can now be passed to `scipy.optimize.linprog` with ``method='highs'``.
`scipy.signal` improvements ====================== - The new window function `scipy.signal.windows.lanczos` was added to compute a Lanczos window, also known as a sinc window.
`scipy.sparse.csgraph` improvements ============================= - the performance of `scipy.sparse.csgraph.dijkstra` has been improved, and star graphs in particular see a marked performance improvement
`scipy.special` improvements ======================= - The new function `scipy.special.powm1`, a ufunc with signature ``powm1(x, y)``, computes ``x**y - 1``. The function avoids the loss of precision that can result when ``y`` is close to 0 or when ``x`` is close to 1. - `scipy.special.erfinv` is now more accurate as it leverages the Boost equivalent under the hood.
`scipy.stats` improvements ===================== - Added `scipy.stats.goodness_of_fit`, a generalized goodness-of-fit test for use with any univariate distribution, any combination of known and unknown parameters, and several choices of test statistic (Kolmogorov-Smirnov, Cramer-von Mises, and Anderson-Darling). - Improved `scipy.stats.bootstrap`: Default method ``'BCa'`` now supports multi-sample statistics. Also, the bootstrap distribution is returned in the result object, and the result object can be passed into the function as parameter ``bootstrap_result`` to add additional resamples or change the confidence interval level and type. - Added maximum spacing estimation to `scipy.stats.fit`. - Added the Poisson means test ("E-test") as `scipy.stats.poisson_means_test`. - Added new sample statistics.
- Added `scipy.stats.contingency.odds_ratio` to compute both the conditional and unconditional odds ratios and corresponding confidence intervals for 2x2 contingency tables. - Added `scipy.stats.directional_stats` to compute sample statistics of n-dimensional directional data. - Added `scipy.stats.expectile`, which generalizes the expected value in the same way as quantiles are a generalization of the median.
- Added new statistical distributions.
- Added `scipy.stats.uniform_direction`, a multivariate distribution to sample uniformly from the surface of a hypersphere. - Added `scipy.stats.random_table`, a multivariate distribution to sample uniformly from m x n contingency tables with provided marginals. - Added `scipy.stats.truncpareto`, the truncated Pareto distribution.
- Improved the ``fit`` method of several distributions.
- `scipy.stats.skewnorm` and `scipy.stats.weibull_min` now use an analytical solution when ``method='mm'``, which also serves a starting guess to improve the performance of ``method='mle'``. - `scipy.stats.gumbel_r` and `scipy.stats.gumbel_l`: analytical maximum likelihood estimates have been extended to the cases in which location or scale are fixed by the user. - Analytical maximum likelihood estimates have been added for `scipy.stats.powerlaw`.
- Improved random variate sampling of several distributions.
- Drawing multiple samples from `scipy.stats.matrix_normal`, `scipy.stats.ortho_group`, `scipy.stats.special_ortho_group`, and `scipy.stats.unitary_group` is faster. - The ``rvs`` method of `scipy.stats.vonmises` now wraps to the interval ``[-np.pi, np.pi]``. - Improved the reliability of `scipy.stats.loggamma` ``rvs`` method for small values of the shape parameter.
- Improved the speed and/or accuracy of functions of several statistical distributions.
- Added `scipy.stats.Covariance` for better speed, accuracy, and user control in multivariate normal calculations. - `scipy.stats.skewnorm` methods ``cdf``, ``sf``, ``ppf``, and ``isf`` methods now use the implementations from Boost, improving speed while maintaining accuracy. The calculation of higher-order moments is also faster and more accurate. - `scipy.stats.invgauss` methods ``ppf`` and ``isf`` methods now use the implementations from Boost, improving speed and accuracy. - `scipy.stats.invweibull` methods ``sf`` and ``isf`` are more accurate for small probability masses. - `scipy.stats.nct` and `scipy.stats.ncx2` now rely on the implementations from Boost, improving speed and accuracy. - Implemented the ``logpdf`` method of `scipy.stats.vonmises` for reliability in extreme tails. - Implemented the ``isf`` method of `scipy.stats.levy` for speed and accuracy. - Improved the robustness of `scipy.stats.studentized_range` for large ``df`` by adding an infinite degree-of-freedom approximation. - Added a parameter ``lower_limit`` to `scipy.stats.multivariate_normal`, allowing the user to change the integration limit from -inf to a desired value. - Improved the robustness of ``entropy`` of `scipy.stats.vonmises` for large concentration values.
- Enhanced `scipy.stats.gaussian_kde`.
- Added `scipy.stats.gaussian_kde.marginal`, which returns the desired marginal distribution of the original kernel density estimate distribution. - The ``cdf`` method of `scipy.stats.gaussian_kde` now accepts a ``lower_limit`` parameter for integrating the PDF over a rectangular region. - Moved calculations for `scipy.stats.gaussian_kde.logpdf` to Cython, improving speed. - The global interpreter lock is released by the ``pdf`` method of `scipy.stats.gaussian_kde` for improved multithreading performance. - Replaced explicit matrix inversion with Cholesky decomposition for speed and accuracy.
- Enhanced the result objects returned by many `scipy.stats` functions
- Added a ``confidence_interval`` method to the result object returned by `scipy.stats.ttest_1samp` and `scipy.stats.ttest_rel`. - The `scipy.stats` functions ``combine_pvalues``, ``fisher_exact``, ``chi2_contingency``, ``median_test`` and ``mood`` now return bunch objects rather than plain tuples, allowing attributes to be accessed by name. - Attributes of the result objects returned by ``multiscale_graphcorr``, ``anderson_ksamp``, ``binomtest``, ``crosstab``, ``pointbiserialr``, ``spearmanr``, ``kendalltau``, and ``weightedtau`` have been renamed to ``statistic`` and ``pvalue`` for consistency throughout `scipy.stats`. Old attribute names are still allowed for backward compatibility. - `scipy.stats.anderson` now returns the parameters of the fitted distribution in a `scipy.stats._result_classes.FitResult` object. - The ``plot`` method of `scipy.stats._result_classes.FitResult` now accepts a ``plot_type`` parameter; the options are ``'hist'`` (histogram, default), ``'qq'`` (Q-Q plot), ``'pp'`` (P-P plot), and ``'cdf'`` (empirical CDF plot). - Kolmogorov-Smirnov tests (e.g. `scipy.stats.kstest`) now return the location (argmax) at which the statistic is calculated and the variant of the statistic used.
- Improved the performance of several `scipy.stats` functions.
- Improved the performance of `scipy.stats.cramervonmises_2samp` and `scipy.stats.ks_2samp` with ``method='exact'``. - Improved the performance of `scipy.stats.siegelslopes`. - Improved the performance of `scipy.stats.mstats.hdquantile_sd`. - Improved the performance of `scipy.stats.binned_statistic_dd` for several NumPy statistics, and binned statistics methods now support complex data.
- Added the ``scramble`` optional argument to `scipy.stats.qmc.LatinHypercube`. It replaces ``centered``, which is now deprecated. - Added a parameter ``optimization`` to all `scipy.stats.qmc.QMCEngine` subclasses to improve characteristics of the quasi-random variates. - Added tie correction to `scipy.stats.mood`. - Added tutorials for resampling methods in `scipy.stats`. - `scipy.stats.bootstrap`, `scipy.stats.permutation_test`, and `scipy.stats.monte_carlo_test` now automatically detect whether the provided ``statistic`` is vectorized, so passing the ``vectorized`` argument explicitly is no longer required to take advantage of vectorized statistics. - Improved the speed of `scipy.stats.permutation_test` for permutation types ``'samples'`` and ``'pairings'``. - Added ``axis``, ``nan_policy``, and masked array support to `scipy.stats.jarque_bera`. - Added the ``nan_policy`` optional argument to `scipy.stats.rankdata`.
*********************** Deprecated features *********************** - `scipy.misc` module and all the methods in ``misc`` are deprecated in v1.10 and will be completely removed in SciPy v2.0.0. Users are suggested to utilize the `scipy.datasets` module instead for the dataset methods. - `scipy.stats.qmc.LatinHypercube` parameter ``centered`` has been deprecated. It is replaced by the ``scramble`` argument for more consistency with other QMC engines. - `scipy.interpolate.interp2d` class has been deprecated. The docstring of the deprecated routine lists recommended replacements.
************************* Expired Deprecations ************************* - There is an ongoing effort to follow through on long-standing deprecations. - The following previously deprecated features are affected:
- Removed ``cond`` & ``rcond`` kwargs in ``linalg.pinv`` - Removed wrappers ``scipy.linalg.blas.{clapack, flapack}`` - Removed ``scipy.stats.NumericalInverseHermite`` and removed ``tol`` & ``max_intervals`` kwargs from ``scipy.stats.sampling.NumericalInverseHermite`` - Removed ``local_search_options`` kwarg frrom ``scipy.optimize.dual_annealing``.
***************** Other changes ***************** - `scipy.stats.bootstrap`, `scipy.stats.permutation_test`, and `scipy.stats.monte_carlo_test` now automatically detect whether the provided ``statistic`` is vectorized by looking for an ``axis`` parameter in the signature of ``statistic``. If an ``axis`` parameter is present in ``statistic`` but should not be relied on for vectorized calls, users must pass option ``vectorized==False`` explicitly. - `scipy.stats.multivariate_normal` will now raise a ``ValueError`` when the covariance matrix is not positive semidefinite, regardless of which method is called.
********* Authors *********
* Name (commits) * h-vetinari (10) * Jelle Aalbers (1) * Alan-Hung (1) + * Tania Allard (7) * Oren Amsalem (1) + * Sven Baars (10) * Balthasar (1) + * Ross Barnowski (1) * Christoph Baumgarten (2) * Peter Bell (2) * Sebastian Berg (1) * Aaron Berk (1) + * boatwrong (1) + * Jake Bowhay (50) * Matthew Brett (4) * Evgeni Burovski (93) * Matthias Bussonnier (6) * Dominic C (2) * Mingbo Cai (1) + * James Campbell (2) + * CJ Carey (4) * cesaregarza (1) + * charlie0389 (1) + * Hood Chatham (5) * Andrew Chin (1) + * Daniel Ching (1) + * Leo Chow (1) + * chris (3) + * John Clow (1) + * cm7S (1) + * cmgodwin (1) + * Christopher Cowden (2) + * Henry Cuzco (2) + * Anirudh Dagar (10) * Hans Dembinski (2) + * Jaiden di Lanzo (24) + * Felipe Dias (1) + * Dieter Werthmüller (1) * Giuseppe Dilillo (1) + * dpoerio (1) + * drpeteb (1) + * Christopher Dupuis (1) + * Jordan Edmunds (1) + * Pieter Eendebak (1) + * Jérome Eertmans (1) + * Fabian Egli (2) + * Sebastian Ehlert (2) + * Kian Eliasi (1) + * Tomohiro Endo (1) + * Stefan Endres (1) * Zeb Engberg (4) + * Jonas Eschle (1) + * Thomas J. Fan (9) * fiveseven (1) + * Neil Flood (1) + * Franz Forstmayr (1) * Sara Fridovich-Keil (1) * David Gilbertson (1) + * Ralf Gommers (251) * Marco Gorelli (2) + * Matt Haberland (381) * Andrew Hawryluk (2) + * Christoph Hohnerlein (2) + * Loïc Houpert (2) + * Shamus Husheer (1) + * ideasrule (1) + * imoiwm (1) + * Lakshaya Inani (1) + * Joseph T. Iosue (1) * iwbc-mzk (1) + * Nathan Jacobi (3) + * Julien Jerphanion (5) * He Jia (1) * jmkuebler (1) + * Johannes Müller (1) + * Vedant Jolly (1) + * Juan Luis Cano Rodríguez (2) * Justin (1) + * jvavrek (1) + * jyuv (2) * Kai Mühlbauer (1) + * Nikita Karetnikov (3) + * Reinert Huseby Karlsen (1) + * kaspar (2) + * Toshiki Kataoka (1) * Robert Kern (3) * Joshua Klein (1) + * Andrew Knyazev (7) * Jozsef Kutas (16) + * Eric Larson (4) * Lechnio (1) + * Antony Lee (2) * Aditya Limaye (1) + * Xingyu Liu (2) * Christian Lorentzen (4) * Loïc Estève (2) * Thibaut Lunet (2) + * Peter Lysakovski (1) * marianasalamoni (2) + * mariprudencio (1) + * Paige Martin (1) + * Arno Marty (1) + * matthewborish (3) + * Damon McDougall (1) * Nicholas McKibben (22) * McLP (1) + * mdmahendri (1) + * Melissa Weber Mendonça (9) * Jarrod Millman (1) * Naoto Mizuno (2) * Shashaank N (1) * Pablo S Naharro (1) + * nboudrie (1) + * Andrew Nelson (52) * Nico Schlömer (1) * NiMlr (1) + * o-alexandre-felipe (1) + * Maureen Ononiwu (1) + * Dimitri Papadopoulos (2) + * partev (1) + * Tirth Patel (10) * Paulius Šarka (1) + * Josef Perktold (1) * Giacomo Petrillo (3) + * Matti Picus (1) * Rafael Pinto (1) + * PKNaveen (1) + * Ilhan Polat (6) * Akshita Prasanth (2) + * Sean Quinn (1) * Tyler Reddy (117) * Martin Reinecke (1) * Ned Richards (1) * Marie Roald (1) + * Sam Rosen (4) + * Pamphile Roy (103) * sabonerune (2) + * Atsushi Sakai (94) * Daniel Schmitz (27) * Anna Scholtz (1) + * Eli Schwartz (11) * serge-sans-paille (2) * JEEVANSHI SHARMA (1) + * ehsan shirvanian (2) + * siddhantwahal (2) * Mathieu Dutour Sikiric (1) + * Sourav Singh (1) * Alexander Soare (1) + * Bjørge Solli (2) + * Scott Staniewicz (1) * Albert Steppi (3) * Thomas Stoeger (1) + * Kai Striega (4) * Tartopohm (1) + * Mamoru TASAKA (2) + * Ewout ter Hoeven (5) * TianyiQ (1) + * Tiger (1) + * Will Tirone (1) * Edgar Andrés Margffoy Tuay (1) + * Dmitry Ulyumdzhiev (1) + * Hari Vamsi (1) + * VitalyChait (1) + * Rik Voorhaar (1) + * Samuel Wallan (4) * Stefan van der Walt (2) * Warren Weckesser (145) * wei2222 (1) + * windows-server-2003 (3) + * Marek Wojciechowski (2) + * Niels Wouda (1) + * WRKampi (1) + * Yeonjoo Yoo (1) + * Rory Yorke (1) * Xiao Yuan (2) + * Meekail Zain (2) + * Fabio Zanini (1) + * Steffen Zeile (1) + * Egor Zemlyanoy (19) * Gavin Zhang (3) +
A total of 180 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete.
**************************** Issues closed for 1.10.0 ****************************