From mailinglists at xgm.de Mon Sep 4 03:26:51 2017 From: mailinglists at xgm.de (Florian Lindner) Date: Mon, 4 Sep 2017 15:26:51 +0800 Subject: [SciPy-User] Creating meshgrid from meshgrid Message-ID: <410fc630-0d05-6240-c458-2fad26bb42ae@xgm.de> Hello, yes, the subject sound a bit weird... I have two arrays of size N (let's say 2 arrays of length 4) that I combine using np.meshgrid xxA, yyA = np.meshgrid(xA, yA) xxB, yyB = np.meshgrid(xB, yB) which gives me two meshes xx.shape = yy.shape = (4,4) which represent a N-dimensional mesh with 16 elements. Now I want to evaluate a function f on every possible pair of N-dimensional points in the grid, resulting in a 16 x 16 matrix: in a flattened notation, pA = (xxA, yyA) f(pA[1]-pB[1]) f(pA[1]-pB[2]) f(pA[1]-pB[3]) ... f(pA[2]-pB[1]) f(pA[2]-pB[2]) f(pA[2]-pB[3]) ... f(pA[3]-pB[1]) f(pA[3]-pB[2]) f(pA[3]-pB[3]) ... . . . Let's say xA = yA = [1,2,3] and xB = yB = [10,20,30] that gives me a mesh A: (1,3) (2,3) (3,3) (1,2) (2,2) (3,2) (1,1) (2,1) (3,1) and a mesh B alike. My result matrix now should be of size 9 x 9: f( (1,3), (10,30) ) f( (2,3), (20,30) ) f( (3,3), (30, 30) ) f( (1,2), (10,20) ) f( (2,2), (20,20) ) f( (3,2), (30, 20) ) ... f always takes two N-dimensional vectors and returns a scalar. I hope I was able to explain what I want to achieve. What is the best way to do that in numpy/scipy? As long as the meshes itself are 1-d I did it like that: mgrid = np.meshgrid([1,2,3], [10,20,30]) A = f( np.abs(mgrid[0] - mgrid[1]) ) Thanks, Florian From chris.barker at noaa.gov Tue Sep 5 19:54:27 2017 From: chris.barker at noaa.gov (Chris Barker) Date: Tue, 5 Sep 2017 16:54:27 -0700 Subject: [SciPy-User] Creating meshgrid from meshgrid In-Reply-To: <410fc630-0d05-6240-c458-2fad26bb42ae@xgm.de> References: <410fc630-0d05-6240-c458-2fad26bb42ae@xgm.de> Message-ID: I'm confused as to what you are trying to do. Meshgrid is used to create a regular grid when you have a vectors of the axes. It support 2 or more dimensions. I have two arrays of size N (let's say 2 arrays of length 4) that I combine > using np.meshgrid > > xxA, yyA = np.meshgrid(xA, yA) > xxB, yyB = np.meshgrid(xB, yB) > > which gives me two meshes > > xx.shape = yy.shape = (4,4) > which represent a N-dimensional mesh with 16 elements. > no -- it represents a 2-dimensional mesh with four nodes in each direction. > Now I want to evaluate a function f on every possible pair of > N-dimensional points in the grid, resulting in a 16 x 16 > matrix: > I think you are looking for a different function than meshgrid. But if you want to evaluate a function on a four dimensional space, you can use meshgrid with four dimensions: xx, yy, zz, tt = meshgrid(x, y, z, t) results = func(xx,yy,zz,tt) note that with numpy's broadcasting, you may not need to use meshgrid at all. Is that what you are looking for? -CHB -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker at noaa.gov -------------- next part -------------- An HTML attachment was scrubbed... URL: From mailinglists at xgm.de Wed Sep 6 00:31:29 2017 From: mailinglists at xgm.de (Florian Lindner) Date: Wed, 6 Sep 2017 12:31:29 +0800 Subject: [SciPy-User] Creating meshgrid from meshgrid In-Reply-To: References: <410fc630-0d05-6240-c458-2fad26bb42ae@xgm.de> Message-ID: <4c6384c6-151b-0f9e-d44d-22601f9b3b8b@xgm.de> Hey, I'm sorry, I realized my posting was confusing, tried to rewrite it: In 1-d I can easily and elegantly achieve what I want, but I'm unable to transfer it to 2-d or even higher dimensional **1-d** I have two 1-d meshes of size N = 3. a = [1, 2, 3] b = [10, 20, 30] now I want to evaluate a function fun on on the difference/norm of all pairs of these meshes: def fun(x): return x mgrid = np.meshgrid( a,b ) A = fun( np.abs(mgrid[0] - mgrid[1] ) ) Result A is of size N x N: array([[ 9, 8, 7], [19, 18, 17], [29, 28, 27]]) which is |b[0]-a[0]| |b[0]-a[1]| |b[0]-a[2]| |b[1]-a[0]| |b[1]-a[1]| |b[1]-a[2]| |b[2]-a[0]| |b[2]-a[1]| |b[2]-a[2]| (only arguments of fun, for sake of brevity) **2-d** Now in 2-d function fun stays the same. I have again two meshes a and b ax = [1, 2, 3] ay = [4, 5, 6] bx = [10, 20, 30] by = [40, 50, 60] a = np.meshgrid(ax, ay) b = np.meshgrid(bx, by) Now, how can I do the same I did with the 1-d meshes above for 2-d and possibly also for higher dimensions? The first row of A: || (10 40) - (1 4) || || (10 40) - (1 5) || || (10 40) - (1 6) || || (10 40) - (2 4) || || (10 40) - (2 5) || || (10 40) - (2 6) || || (10 40) - (3 4) || || (10 40) - (3 5) || || (10 40) - (3 6) || (everything is jus the first row of A, again only arguments of fun) The result mesh should have the size N * N x N * N. I tried to create the coordinates of a and b ca = np.array(list(zip(a[0].flatten(), a[1].flatten()))) cb = np.array(list(zip(b[0].flatten(), b[1].flatten()))) And create a meshgrid from that: mgrid = np.meshgrid([ca], [cb]) but alone the dimensionality does not fit (18 instead of 9). I hope I was now able to better get across what I want. Thanks! Florian Am 06.09.2017 um 07:54 schrieb Chris Barker: > I'm confused as to what you are trying to do. > > Meshgrid is used to create a regular grid when you have a vectors of the axes. > > It support 2 or more dimensions. > > I have two arrays of size N (let's say 2 arrays of length 4) that I combine using np.meshgrid > > xxA, yyA = np.meshgrid(xA, yA) > xxB, yyB = np.meshgrid(xB, yB) > > which gives me two meshes > > xx.shape = yy.shape = (4,4) > which represent a N-dimensional mesh with 16 elements. > > > no -- it represents a 2-dimensional mesh with four nodes in each direction. > ? > > Now I want to evaluate a function f on every possible pair of N-dimensional points in the grid, resulting in a 16 x 16 > matrix: > > > I think you are looking for a different function than meshgrid. > > But if you want to evaluate a function on a four dimensional space, you can use meshgrid with four dimensions: > > xx, yy, zz, tt = meshgrid(x, y, z, t) > > results = func(xx,yy,zz,tt) > > note that with numpy's broadcasting, you may not need to use meshgrid at all. > > Is that what you are looking for? > > -CHB > > > -- > > Christopher Barker, Ph.D. > Oceanographer > > Emergency Response Division > NOAA/NOS/OR&R ? ? ? ? ? ?(206) 526-6959?? voice > 7600 Sand Point Way NE ??(206) 526-6329?? fax > Seattle, WA ?98115 ? ? ??(206) 526-6317?? main reception > > Chris.Barker at noaa.gov > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > From pav at iki.fi Wed Sep 6 04:05:54 2017 From: pav at iki.fi (Pauli Virtanen) Date: Wed, 6 Sep 2017 10:05:54 +0200 Subject: [SciPy-User] Creating meshgrid from meshgrid In-Reply-To: <4c6384c6-151b-0f9e-d44d-22601f9b3b8b@xgm.de> References: <410fc630-0d05-6240-c458-2fad26bb42ae@xgm.de> <4c6384c6-151b-0f9e-d44d-22601f9b3b8b@xgm.de> Message-ID: <1d31e4e5-fbef-e360-e903-56247562bf74@iki.fi> Florian Lindner kirjoitti 06.09.2017 klo 06:31: [clip] > def fun(x): > return x > > mgrid = np.meshgrid( a,b ) > A = fun( np.abs(mgrid[0] - mgrid[1] ) ) > > Result A is of size N x N: > > array([[ 9, 8, 7], > [19, 18, 17], > [29, 28, 27]]) Don't use meshgrid, use broadcasting: from numpy import newaxis ax = np.array([0, 1, 2]) ay = np.array([4, 5]) bx = np.array([7, 8, 9, 10]) by = np.array([11, 12, 13, 14]) def fun(x1, y1, x2, y2): return abs(x1 - x2) + abs(y1 - y2) d = fun(ax[:,newaxis,newaxis,newaxis], ay[newaxis,:,newaxis,newaxis], bx[newaxis,newaxis,:,newaxis], by[newaxis,newaxis,newaxis,:]) or p1 = np.array([[0,1], [1,3], [5,6], [7,9]]) p2 = np.array([[3,6], [2,7], [9,2], [1,3]]) def fun(a, b): return np.hypot(a[...,0] - b[...,0], a[...,1] - b[...,1]) d = fun(p1[:,newaxis,:], p2[newaxis,:,:]) depending on what it is that you actually want to evaluate. -- Pauli Virtanen From mailinglists at xgm.de Thu Sep 7 00:54:53 2017 From: mailinglists at xgm.de (Florian Lindner) Date: Thu, 7 Sep 2017 12:54:53 +0800 Subject: [SciPy-User] Creating meshgrid from meshgrid In-Reply-To: <1d31e4e5-fbef-e360-e903-56247562bf74@iki.fi> References: <410fc630-0d05-6240-c458-2fad26bb42ae@xgm.de> <4c6384c6-151b-0f9e-d44d-22601f9b3b8b@xgm.de> <1d31e4e5-fbef-e360-e903-56247562bf74@iki.fi> Message-ID: <40e283fc-1bbe-92c7-956d-fa6f099ae6ea@xgm.de> Hi, thanks for your input! Am 06.09.2017 um 16:05 schrieb Pauli Virtanen: > Florian Lindner kirjoitti 06.09.2017 klo 06:31: > [clip] >> ???? def fun(x): >> ???????? return x >> >> ???? mgrid = np.meshgrid( a,b ) >> ???? A = fun( np.abs(mgrid[0] - mgrid[1] ) ) >> >> Result A is of size N x N: >> >> ???? array([[ 9,? 8,? 7], >> ??????????? [19, 18, 17], >> ??????????? [29, 28, 27]]) > > Don't use meshgrid, use broadcasting: > > from numpy import newaxis > > ax = np.array([0, 1, 2]) > ay = np.array([4, 5]) > bx = np.array([7, 8, 9, 10]) > by = np.array([11, 12, 13, 14]) > > def fun(x1, y1, x2, y2): > ??? return abs(x1 - x2) + abs(y1 - y2) > > d = fun(ax[:,newaxis,newaxis,newaxis], > ??????? ay[newaxis,:,newaxis,newaxis], > ??????? bx[newaxis,newaxis,:,newaxis], > ??????? by[newaxis,newaxis,newaxis,:]) Ok, this gives me what I want after a d.reshape(6, 16). However, it is very tied to two dimensions and seem to not easily generalizable to more (or less) dimensions. > or > > p1 = np.array([[0,1], [1,3], [5,6], [7,9]]) > p2 = np.array([[3,6], [2,7], [9,2], [1,3]]) > > def fun(a, b): > ??? return np.hypot(a[...,0] - b[...,0], a[...,1] - b[...,1]) > > d = fun(p1[:,newaxis,:], p2[newaxis,:,:]) > > depending on what it is that you actually want to evaluate. That returns d of shape (4, 4), where I expect (16, 16). I try to reformulate my problem: From: ax = np.array([1,2,3]) ay = np.array([3,4,5]) a = np.meshgrid(ax, ay) bx = np.array([10,20,30]) by = np.array([30,40,50]) b = np.meshgrid(bx, by) I can build the coordinates of all points in the meshes: ca = np.array(list(zip(a[0].flatten(), a[1].flatten()))) cb = np.array(list(zip(b[0].flatten(), b[1].flatten()))) (in 1-d it will be just like ca = np.linspace(...)) just like your p1 and p2. Now: A = np.zeros([len(ca), len(cb)]) for i, x in enumerate(ca): for j, y in enumerate(cb): A[i, j] = func(np.linalg.norm(x-y)) gives me exactly what I want and should work for any number of dimensions in ca/cb. , albeit in a very un-numpythonic and probably slow way. You have any hints on how to improve that? Best Thanks, Florian From ralf.gommers at gmail.com Sun Sep 17 06:48:35 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Sun, 17 Sep 2017 22:48:35 +1200 Subject: [SciPy-User] ANN: SciPy 1.0 beta release Message-ID: Hi all, I'm excited to be able to announce the availability of the first beta release of Scipy 1.0. This is a big release, and a version number that has been 16 years in the making. It contains a few more deprecations and backwards incompatible changes than an average release. Therefore please do test it on your own code, and report any issues on the Github issue tracker or on the scipy-dev mailing list. Sources: https://github.com/scipy/scipy/releases/tag/v1.0.0b1 Binary wheels: will follow tomorrow, I'll announce those when ready (TravisCI is under maintenance right now) Thanks to everyone who contributed to this release! Ralf Release notes (full notes including authors, closed issued and merged PRs at the Github Releases link above): ========================== SciPy 1.0.0 Release Notes ========================== .. note:: Scipy 1.0.0 is not released yet! .. contents:: SciPy 1.0.0 is the culmination of 8 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Moreover, our development attention will now shift to bug-fix releases on the 1.0.x branch, and on adding new features on the master branch. Some of the highlights of this release are: - Major build improvements. Windows wheels are available on PyPI for the first time, and continuous integration has been set up on Windows and OS X in addition to Linux. - A set of new ODE solvers and a unified interface to them (`scipy.integrate.solve_ivp`). - Two new trust region optimizers and a new linear programming method, with improved performance compared to what `scipy.optimize` offered previously. - Many new BLAS and LAPACK functions were wrapped. The BLAS wrappers are now complete. This release requires Python 2.7 or 3.4+ and NumPy 1.8.2 or greater. This is also the last release to support LAPACK 3.1.x - 3.3.x. Moving the lowest supported LAPACK version to >3.2.x was long blocked by Apple Accelerate providing the LAPACK 3.2.1 API. We have decided that it's time to either drop Accelerate or, if there is enough interest, provide shims for functions added in more recent LAPACK versions so it can still be used. New features ============ `scipy.cluster` improvements ---------------------------- `scipy.cluster.hierarchy.optimal_leaf_ordering`, a function to reorder a linkage matrix to minimize distances between adjacent leaves, was added. `scipy.fftpack` improvements ---------------------------- N-dimensional versions of the discrete sine and cosine transforms and their inverses were added as ``dctn``, ``idctn``, ``dstn`` and ``idstn``. `scipy.integrate` improvements ------------------------------ A set of new ODE solvers have been added to `scipy.integrate`. The convenience function `scipy.integrate.solve_ivp` allows uniform access to all solvers. The individual solvers (``RK23``, ``RK45``, ``Radau``, ``BDF`` and ``LSODA``) can also be used directly. `scipy.linalg` improvements ---------------------------- The BLAS wrappers in `scipy.linalg.blas` have been completed. Added functions are ``*gbmv``, ``*hbmv``, ``*hpmv``, ``*hpr``, ``*hpr2``, ``*spmv``, ``*spr``, ``*tbmv``, ``*tbsv``, ``*tpmv``, ``*tpsv``, ``*trsm``, ``*trsv``, ``*sbmv``, ``*spr2``, Wrappers for the LAPACK functions ``*gels``, ``*stev``, ``*sytrd``, ``*hetrd``, ``*sytf2``, ``*hetrf``, ``*sytrf``, ``*sycon``, ``*hecon``, ``*gglse``, ``*stebz``, ``*stemr``, ``*sterf``, and ``*stein`` have been added. The function `scipy.linalg.subspace_angles` has been added to compute the subspace angles between two matrices. The function `scipy.linalg.clarkson_woodruff_transform` has been added. It finds low-rank matrix approximation via the Clarkson-Woodruff Transform. The functions `scipy.linalg.eigh_tridiagonal` and `scipy.linalg.eigvalsh_tridiagonal`, which find the eigenvalues and eigenvectors of tridiagonal hermitian/symmetric matrices, were added. `scipy.ndimage` improvements ---------------------------- Support for homogeneous coordinate transforms has been added to `scipy.ndimage.affine_transform`. The ``ndimage`` C code underwent a significant refactoring, and is now a lot easier to understand and maintain. `scipy.optimize` improvements ----------------------------- The methods ``trust-region-exact`` and ``trust-krylov`` have been added to the function `scipy.optimize.minimize`. These new trust-region methods solve the subproblem with higher accuracy at the cost of more Hessian factorizations (compared to dogleg) or more matrix vector products (compared to ncg) but usually require less nonlinear iterations and are able to deal with indefinite Hessians. They seem very competitive against the other Newton methods implemented in scipy. `scipy.optimize.linprog` gained an interior point method. Its performance is superior (both in accuracy and speed) to the older simplex method. `scipy.signal` improvements --------------------------- An argument ``fs`` (sampling frequency) was added to the following functions: ``firwin``, ``firwin2``, ``firls``, and ``remez``. This makes these functions consistent with many other functions in `scipy.signal` in which the sampling frequency can be specified. `scipy.signal.freqz` has been sped up significantly for FIR filters. `scipy.sparse` improvements --------------------------- Iterating over and slicing of CSC and CSR matrices is now faster by up to ~35%. The ``tocsr`` method of COO matrices is now several times faster. The ``diagonal`` method of sparse matrices now takes a parameter, indicating which diagonal to return. `scipy.sparse.linalg` improvements ---------------------------------- A new iterative solver for large-scale nonsymmetric sparse linear systems, `scipy.sparse.linalg.gcrotmk`, was added. It implements ``GCROT(m,k)``, a flexible variant of ``GCROT``. `scipy.sparse.linalg.lsmr` now accepts an initial guess, yielding potentially faster convergence. SuperLU was updated to version 5.2.1. `scipy.spatial` improvements ---------------------------- Many distance metrics in `scipy.spatial.distance` gained support for weights. The signatures of `scipy.spatial.distance.pdist` and `scipy.spatial.distance.cdist` were changed to ``*args, **kwargs`` in order to support a wider range of metrics (e.g. string-based metrics that need extra keywords). Also, an optional ``out`` parameter was added to ``pdist`` and ``cdist`` allowing the user to specify where the resulting distance matrix is to be stored `scipy.stats` improvements -------------------------- The methods ``cdf`` and ``logcdf`` were added to `scipy.stats.multivariate_normal`, providing the cumulative distribution function of the multivariate normal distribution. New statistical distance functions were added, namely `scipy.stats.wasserstein_distance` for the first Wasserstein distance and `scipy.stats.energy_distance` for the energy distance. Deprecated features =================== The following functions in `scipy.misc` are deprecated: ``bytescale``, ``fromimage``, ``imfilter``, ``imread``, ``imresize``, ``imrotate``, ``imsave``, ``imshow`` and ``toimage``. Most of those functions have unexpected behavior (like rescaling and type casting image data without the user asking for that). Other functions simply have better alternatives. ``scipy.interpolate.interpolate_wrapper`` and all functions in that submodule are deprecated. This was a never finished set of wrapper functions which is not relevant anymore. The ``fillvalue`` of `scipy.signal.convolve2d` will be cast directly to the dtypes of the input arrays in the future and checked that it is a scalar or an array with a single element. Backwards incompatible changes ============================== The following deprecated functions have been removed from `scipy.stats`: ``betai``, ``chisqprob``, ``f_value``, ``histogram``, ``histogram2``, ``pdf_fromgamma``, ``signaltonoise``, ``square_of_sums``, ``ss`` and ``threshold``. The following deprecated functions have been removed from `scipy.stats.mstats`: ``betai``, ``f_value_wilks_lambda``, ``signaltonoise`` and ``threshold``. The deprecated ``a`` and ``reta`` keywords have been removed from `scipy.stats.shapiro`. The deprecated functions ``sparse.csgraph.cs_graph_components`` and ``sparse.linalg.symeig`` have been removed from `scipy.sparse`. The following deprecated keywords have been removed in `scipy.sparse.linalg`: ``drop_tol`` from ``splu``, and ``xtype`` from ``bicg``, ``bicgstab``, ``cg``, ``cgs``, ``gmres``, ``qmr`` and ``minres``. The deprecated functions ``expm2`` and ``expm3`` have been removed from `scipy.linalg`. The deprecated keyword ``q`` was removed from `scipy.linalg.expm`. And the deprecated submodule ``linalg.calc_lwork`` was removed. The deprecated functions ``C2K``, ``K2C``, ``F2C``, ``C2F``, ``F2K`` and ``K2F`` have been removed from `scipy.constants`. The deprecated ``ppform`` class was removed from `scipy.interpolate`. The deprecated keyword ``iprint`` was removed from `scipy.optimize.fmin_cobyla`. The default value for the ``zero_phase`` keyword of `scipy.signal.decimate` has been changed to True. The ``kmeans`` and ``kmeans2`` functions in `scipy.cluster.vq` changed the method used for random initialization, so using a fixed random seed will not necessarily produce the same results as in previous versions. `scipy.special.gammaln` does not accept complex arguments anymore. The deprecated functions ``sph_jn``, ``sph_yn``, ``sph_jnyn``, ``sph_in``, ``sph_kn``, and ``sph_inkn`` have been removed. Users should instead use the functions ``spherical_jn``, ``spherical_yn``, ``spherical_in``, and ``spherical_kn``. Be aware that the new functions have different signatures. The cross-class properties of `scipy.signal.lti` systems have been removed. The following properties/setters have been removed: Name - (accessing/setting has been removed) - (setting has been removed) * StateSpace - (``num``, ``den``, ``gain``) - (``zeros``, ``poles``) * TransferFunction (``A``, ``B``, ``C``, ``D``, ``gain``) - (``zeros``, ``poles``) * ZerosPolesGain (``A``, ``B``, ``C``, ``D``, ``num``, ``den``) - () ``signal.freqz(b, a)`` with ``b`` or ``a`` >1-D raises a ``ValueError``. This was a corner case for which it was unclear that the behavior was well-defined. The method ``var`` of `scipy.stats.dirichlet` now returns a scalar rather than an ndarray when the length of alpha is 1. Other changes ============= SciPy now has a formal governance structure. It consists of a BDFL (Pauli Virtanen) and a Steering Committee. See `the governance document < https://github.com/scipy/scipy/blob/master/doc/source/dev/governance/governance.rst >`_ for details. It is now possible to build SciPy on Windows with MSVC + gfortran! Continuous integration has been set up for this build configuration on Appveyor, building against OpenBLAS. Continuous integration for OS X has been set up on TravisCI. The SciPy test suite has been migrated from ``nose`` to ``pytest``. ``scipy/_distributor_init.py`` was added to allow redistributors of SciPy to add custom code that needs to run when importing SciPy (e.g. checks for hardware, DLL search paths, etc.). Support for PEP 518 (specifying build system requirements) was added - see ``pyproject.toml`` in the root of the SciPy repository. In order to have consistent function names, the function ``scipy.linalg.solve_lyapunov`` is renamed to `scipy.linalg.solve_continuous_lyapunov`. The old name is kept for backwards-compatibility. -------------- next part -------------- An HTML attachment was scrubbed... URL: From jh at physics.ucf.edu Sun Sep 17 09:16:27 2017 From: jh at physics.ucf.edu (Joe Harrington) Date: Sun, 17 Sep 2017 09:16:27 -0400 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: (scipy-user-request@python.org) Message-ID: On Sun, 17 Sep 2017 22:48:35 +1200 Ralf Gommers wrote: > I'm excited to be able to announce the availability of the first beta > release of Scipy 1.0. > [I won't quote the entire release notes!] Awesome news, looooong in the making! The number of deprecations and removals is understandably large. I think we need a centralized resource for navigating deprecations, removals, and changes. It could be a simple web page, or a more involved database. It should list the change, give or link to the rationale, and state the date and package version number. Most importantly, it should give the suggested replacement(s). Googling "scipy deprecation" found only this page, plus a bunch of release notes and discussions. The page was last updated on Jan 16, 2017, and is limited to C-API deprecations: https://docs.scipy.org/doc/numpy-dev/reference/c-api.deprecations.html Even once such a document/page/searchable database exists, many will be unaware of it, or not know where to find it, so referring to it at the top of any list of deprecations/removals would be good. Once it exists, deprecations could be entered before being implemented, and deprecation warnings could refer to an index number in this resource, where the user could go for more information on how to fix the issue. Thanks, --jh-- From charlesr.harris at gmail.com Sun Sep 17 11:32:15 2017 From: charlesr.harris at gmail.com (Charles R Harris) Date: Sun, 17 Sep 2017 09:32:15 -0600 Subject: [SciPy-User] [Numpy-discussion] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On Sun, Sep 17, 2017 at 4:48 AM, Ralf Gommers wrote: > Hi all, > > I'm excited to be able to announce the availability of the first beta > release of Scipy 1.0. This is a big release, and a version number that > has been 16 years in the making. It contains a few more deprecations and > backwards incompatible changes than an average release. Therefore please do > test it on your own code, and report any issues on the Github issue tracker > or on the scipy-dev mailing list. > > Sources: https://github.com/scipy/scipy/releases/tag/v1.0.0b1 > Binary wheels: will follow tomorrow, I'll announce those when ready > (TravisCI is under maintenance right now) > > Thanks to everyone who contributed to this release! > Congratulations to all, and an extra congratulations to Matthew and everyone else involved in getting the scipy wheels building on all the supported platforms. For those unfamiliar with the history, Ralf became release manager for NumPy 1.4.1 back in early 2010 and switched to full time SciPy release manager in 2011. It has been a long, productive, seven years. Chuck > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Mon Sep 18 04:58:28 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Mon, 18 Sep 2017 20:58:28 +1200 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On Mon, Sep 18, 2017 at 1:16 AM, Joe Harrington wrote: > On Sun, 17 Sep 2017 22:48:35 +1200 Ralf Gommers > wrote: > > > I'm excited to be able to announce the availability of the first beta > > release of Scipy 1.0. > > > [I won't quote the entire release notes!] > > Awesome news, looooong in the making! > > The number of deprecations and removals is understandably large. > I think we need a centralized resource for navigating deprecations, > removals, and changes. It could be a simple web page, or a more > involved database. It should list the change, give or link to the > rationale, and state the date and package version number. Most > importantly, it should give the suggested replacement(s). > > Googling "scipy deprecation" found only this page, plus a bunch of > release notes and discussions. The page was last updated on Jan 16, > 2017, and is limited to C-API deprecations: > > https://docs.scipy.org/doc/numpy-dev/reference/c-api.deprecations.html > > Even once such a document/page/searchable database exists, many will be > unaware of it, or not know where to find it, so referring to it at the > top of any list of deprecations/removals would be good. > > Once it exists, deprecations could be entered before being implemented, > and deprecation warnings could refer to an index number in this > resource, where the user could go for more information on how to fix the > issue. > Thanks for your thoughts Joe! I'm not sure a database is the way to go, but you make a good point. We have to guide users a bit more in what to do about deprecations. I think clear instructions about upgrading in both the html docs and release announcements would be useful. I don't think there's many users who run ``python -Wd my_code.py`` .(making depreacations that affect your code visible) .... Ralf > Thanks, > > --jh-- > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Mon Sep 18 05:59:15 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Mon, 18 Sep 2017 21:59:15 +1200 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On Sun, Sep 17, 2017 at 10:48 PM, Ralf Gommers wrote: > Hi all, > > I'm excited to be able to announce the availability of the first beta > release of Scipy 1.0. This is a big release, and a version number that > has been 16 years in the making. It contains a few more deprecations and > backwards incompatible changes than an average release. Therefore please do > test it on your own code, and report any issues on the Github issue tracker > or on the scipy-dev mailing list. > > Sources: https://github.com/scipy/scipy/releases/tag/v1.0.0b1 > Binary wheels: will follow tomorrow, I'll announce those when ready > (TravisCI is under maintenance right now) > Binary wheels for Windows, Linux and OS X (for all supported Python versions, 32-bit and 64-bit) can be found at http://wheels.scipy.org. To install directly with pip: pip install scipy=='1.0.0b1' -f http://wheels.scipy.org --trusted-host wheels.scipy.org (add --user and/or --upgrade as required to that command). Alternatively, just download the wheel you need and do `pip install scipy-1.0.0b1-.whl`. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Sep 18 06:07:12 2017 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 18 Sep 2017 11:07:12 +0100 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On 18 September 2017 at 10:59, Ralf Gommers wrote: > Binary wheels for Windows, Linux and OS X (for all supported Python > versions, 32-bit and 64-bit) can be found at http://wheels.scipy.org. To > install directly with pip: > > pip install scipy=='1.0.0b1' -f http://wheels.scipy.org > --trusted-host wheels.scipy.org > I don't want to criticise the hard work that has gone into making this available, but I'm disappointed that we're telling people to install software over an insecure HTTP connection. Can the wheels not be uploaded to PyPI? -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Mon Sep 18 06:11:09 2017 From: matthew.brett at gmail.com (Matthew Brett) Date: Mon, 18 Sep 2017 11:11:09 +0100 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: Hi, On Mon, Sep 18, 2017 at 11:07 AM, Thomas Kluyver wrote: > On 18 September 2017 at 10:59, Ralf Gommers wrote: >> >> Binary wheels for Windows, Linux and OS X (for all supported Python >> versions, 32-bit and 64-bit) can be found at http://wheels.scipy.org. To >> install directly with pip: >> >> pip install scipy=='1.0.0b1' -f http://wheels.scipy.org --trusted-host >> wheels.scipy.org > > > I don't want to criticise the hard work that has gone into making this > available, but I'm disappointed that we're telling people to install > software over an insecure HTTP connection. I personally prefer the following recipe: pip install -f https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com scipy=='1.0.0b1' > Can the wheels not be uploaded to PyPI? Sounds like a good idea. I can do that - any objections? Cheers, Matthew From ralf.gommers at gmail.com Mon Sep 18 06:12:53 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Mon, 18 Sep 2017 22:12:53 +1200 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On Mon, Sep 18, 2017 at 10:07 PM, Thomas Kluyver wrote: > On 18 September 2017 at 10:59, Ralf Gommers > wrote: > >> Binary wheels for Windows, Linux and OS X (for all supported Python >> versions, 32-bit and 64-bit) can be found at http://wheels.scipy.org. To >> install directly with pip: >> >> pip install scipy=='1.0.0b1' -f http://wheels.scipy.org >> --trusted-host wheels.scipy.org >> > > I don't want to criticise the hard work that has gone into making this > available, but I'm disappointed that we're telling people to install > software over an insecure HTTP connection. > > Can the wheels not be uploaded to PyPI? > I thought we'd never done that before, but I'm wrong: https://pypi.python.org/pypi/scipy/0.18.0rc2 So yes, I think. Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Mon Sep 18 06:14:34 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Mon, 18 Sep 2017 22:14:34 +1200 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On Mon, Sep 18, 2017 at 10:11 PM, Matthew Brett wrote: > Hi, > > On Mon, Sep 18, 2017 at 11:07 AM, Thomas Kluyver wrote: > > On 18 September 2017 at 10:59, Ralf Gommers > wrote: > >> > >> Binary wheels for Windows, Linux and OS X (for all supported Python > >> versions, 32-bit and 64-bit) can be found at http://wheels.scipy.org. > To > >> install directly with pip: > >> > >> pip install scipy=='1.0.0b1' -f http://wheels.scipy.org > --trusted-host > >> wheels.scipy.org > > > > > > I don't want to criticise the hard work that has gone into making this > > available, but I'm disappointed that we're telling people to install > > software over an insecure HTTP connection. > > I personally prefer the following recipe: > > pip install -f https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a > 43.ssl.cf2.rackcdn.com > scipy=='1.0.0b1' > > > Can the wheels not be uploaded to PyPI? > > Sounds like a good idea. I can do that - any objections? > That would be helpful Matthew, I'm about to sign off for today. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Mon Sep 18 06:36:20 2017 From: matthew.brett at gmail.com (Matthew Brett) Date: Mon, 18 Sep 2017 11:36:20 +0100 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: Hi, On Mon, Sep 18, 2017 at 11:14 AM, Ralf Gommers wrote: > > > On Mon, Sep 18, 2017 at 10:11 PM, Matthew Brett > wrote: >> >> Hi, >> >> On Mon, Sep 18, 2017 at 11:07 AM, Thomas Kluyver wrote: >> > On 18 September 2017 at 10:59, Ralf Gommers >> > wrote: >> >> >> >> Binary wheels for Windows, Linux and OS X (for all supported Python >> >> versions, 32-bit and 64-bit) can be found at http://wheels.scipy.org. >> >> To >> >> install directly with pip: >> >> >> >> pip install scipy=='1.0.0b1' -f http://wheels.scipy.org >> >> --trusted-host >> >> wheels.scipy.org >> > >> > >> > I don't want to criticise the hard work that has gone into making this >> > available, but I'm disappointed that we're telling people to install >> > software over an insecure HTTP connection. >> >> I personally prefer the following recipe: >> >> pip install -f >> https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a43.ssl.cf2.rackcdn.com >> scipy=='1.0.0b1' >> >> > Can the wheels not be uploaded to PyPI? >> >> Sounds like a good idea. I can do that - any objections? > > > That would be helpful Matthew, I'm about to sign off for today. Done - new instructions for testing: pip install --pre --upgrade scipy Cheers, Matthew From ralf.gommers at gmail.com Tue Sep 19 05:10:15 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Tue, 19 Sep 2017 21:10:15 +1200 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On Mon, Sep 18, 2017 at 10:36 PM, Matthew Brett wrote: > Hi, > > On Mon, Sep 18, 2017 at 11:14 AM, Ralf Gommers > wrote: > > > > > > On Mon, Sep 18, 2017 at 10:11 PM, Matthew Brett > > > wrote: > >> > >> Hi, > >> > >> On Mon, Sep 18, 2017 at 11:07 AM, Thomas Kluyver > wrote: > >> > On 18 September 2017 at 10:59, Ralf Gommers > >> > wrote: > >> >> > >> >> Binary wheels for Windows, Linux and OS X (for all supported Python > >> >> versions, 32-bit and 64-bit) can be found at http://wheels.scipy.org > . > >> >> To > >> >> install directly with pip: > >> >> > >> >> pip install scipy=='1.0.0b1' -f http://wheels.scipy.org > >> >> --trusted-host > >> >> wheels.scipy.org > >> > > >> > > >> > I don't want to criticise the hard work that has gone into making this > >> > available, but I'm disappointed that we're telling people to install > >> > software over an insecure HTTP connection. > >> > >> I personally prefer the following recipe: > >> > >> pip install -f > >> https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a4 > 3.ssl.cf2.rackcdn.com > >> scipy=='1.0.0b1' > >> > >> > Can the wheels not be uploaded to PyPI? > >> > >> Sounds like a good idea. I can do that - any objections? > > > > > > That would be helpful Matthew, I'm about to sign off for today. > > Done - new instructions for testing: > > pip install --pre --upgrade scipy > Thanks Matthew! Replying to all lists with the better install instructions. Cheers, Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From cimrman3 at ntc.zcu.cz Tue Sep 19 09:23:23 2017 From: cimrman3 at ntc.zcu.cz (Robert Cimrman) Date: Tue, 19 Sep 2017 15:23:23 +0200 Subject: [SciPy-User] ANN: SfePy 2017.3 Message-ID: <300388fd-de7f-ad26-248a-e81de1f18240@ntc.zcu.cz> I am pleased to announce release 2017.3 of SfePy. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (limited support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: https://mail.python.org/mm3/mailman3/lists/sfepy.python.org/ Git (source) repository, issue tracker: https://github.com/sfepy/sfepy Highlights of this release -------------------------- - support preconditioning in SciPy and PyAMG based linear solvers - user-defined preconditioners for PETSc linear solvers - parallel multiscale (macro-micro) homogenization-based computations - improved tutorial and installation instructions For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Cheers, Robert Cimrman --- Contributors to this release in alphabetical order: Robert Cimrman Lubos Kejzlar Vladimir Lukes Matyas Novak From simo.bolognini at gmail.com Fri Sep 22 03:52:09 2017 From: simo.bolognini at gmail.com (Simone Bolognini) Date: Fri, 22 Sep 2017 09:52:09 +0200 Subject: [SciPy-User] Diagonalization of a tridiagonal, symmetric, sparse matrix with scipy.sparse.linalg.eigsh() In-Reply-To: References: Message-ID: Hi everyone! Here's my problem. I have an NxN symmetric and tridiagonal matrix computed by a Python code and I want to diagonalize it. In the specific case I'm dealing with N = 6000, but the matrix can become larger. Since it is sparse, I assumed the best way to diagonalize it was to use the algorithm scipy.sparse.linalg.eigsh(), which performed extremely good with other sparse and symmetric matrices (not tridiagonal ones, though) I worked with. In particular, since I need only the low lying part of the spectrum, I'm specifying k=2 and which='SM' in the function. However, in this case this algorithm seems not to work, since after approximately 20 minutes of computation I get the following error: ArpackNoConvergence: ARPACK error -1: No convergence (60001 iterations, 0/2 eigenvectors converged) Why is this happening? Is it a problem related to some properties of tridiagonal matrices? What other scipy routine can I use in order to diagonalize my matrix in an efficient way? Here's a minimal code to reproduce my error: import scipy.sparse.linalg as slimport numpy as np dim = 6000 a = np.empty( dim - 1 ) a.fill( 1. ) diag_up = np.diag( a, 1 ) diag_bot = np.diag( a, -1 ) b = np.empty( dim ) b.fill( 1. ) mat = np.diag( b ) + diag_up + diag_bot v, w = sl.eigsh(mat, 2, which = 'SM') On my pc the construction of the matrix mat takes 364ms, while only the last line in which the diagonalization is performed gives the reported error. Thanks a lot for the support. Simone -------------- next part -------------- An HTML attachment was scrubbed... URL: From ljmamoreira at gmail.com Fri Sep 22 10:11:37 2017 From: ljmamoreira at gmail.com (Luis JM Amoreira) Date: Fri, 22 Sep 2017 15:11:37 +0100 Subject: [SciPy-User] Diagonalization of a tridiagonal, symmetric, sparse matrix with scipy.sparse.linalg.eigsh() In-Reply-To: References: Message-ID: Hi Try scipy.linalg.solve_banded. You have to store the matrix in diagonal ordered form. Take a random matrix example: In [1]: import scipy.linalg In [2]: m=scipy.rand(3,6000) In [3]: m[0,0]=0.; m[-1,-1]=0. In [4]: b=scipy.rand(6000) In [5]: %timeit scipy.linalg.solve_banded((1,1),m,b) The slowest run took 49.27 times longer than the fastest. This could mean that an intermediate result is being cached. 1000 loops, best of 3: 262 ?s per loop In steps 2 and 3 I create the diagonal ordered form. Each diagonal of the coefficient matrix (I considered a random tridiagonal 6000x6000 matrix) is stored in each line of matrix m; zero is taken for the first element of the upper diagonal and the last element of the lower diagonal. Hope this helps, but your problem may be harder than my ranom example... Regards Ze Amoreira On 09/22/2017 08:52 AM, Simone Bolognini wrote: > Hi everyone! Here's my problem. > > I have an NxN symmetric and tridiagonal matrix computed by a Python code > and I want to diagonalize it. > > In the specific case I'm dealing with |N = 6000|, but the matrix can > become larger. Since it is sparse, I assumed the best way to diagonalize > it was to use the algorithm |scipy.sparse.linalg.eigsh()|, which > performed extremely good with other sparse and symmetric matrices (not > tridiagonal ones, though) I worked with. In particular, since I need > only the low lying part of the spectrum, I'm specifying |k=2|?and > |which='SM'|?in the function. > > However, in this case this algorithm seems not to work, since after > approximately 20 minutes of computation I get the following error: > > ArpackNoConvergence: ARPACK error -1: No convergence (60001 > iterations, 0/2 eigenvectors converged) > > Why is this happening? Is it a problem related to some properties of > tridiagonal matrices? What other scipy routine can I use in order to > diagonalize my matrix in an efficient way? > > Here's a minimal code to reproduce my error: > > |importscipy.sparse.linalg assl importnumpy asnp dim =6000a =np.empty(dim > -1)a.fill(1.)diag_up =np.diag(a,1)diag_bot =np.diag(a,-1)b =np.empty(dim > )b.fill(1.)mat =np.diag(b )+diag_up +diag_bot v,w =sl.eigsh(mat,2,which > ='SM')| > > On my pc the construction of the matrix mat takes 364ms, while only the > last line in which the diagonalization is performed gives the reported > error. > > > Thanks a lot for the support. > > Simone > > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > From pav at iki.fi Fri Sep 22 14:24:15 2017 From: pav at iki.fi (Pauli Virtanen) Date: Fri, 22 Sep 2017 20:24:15 +0200 Subject: [SciPy-User] Diagonalization of a tridiagonal, symmetric, sparse matrix with scipy.sparse.linalg.eigsh() In-Reply-To: References: Message-ID: <1506104655.2416.3.camel@iki.fi> pe, 2017-09-22 kello 09:52 +0200, Simone Bolognini kirjoitti: [clip] > However, in this case this algorithm seems not to work, since after > approximately 20 minutes of computation I get the following error: > > ArpackNoConvergence: ARPACK error -1: No convergence (60001 > iterations, 0/2 > eigenvectors converged) > > Why is this happening? Is it a problem related to some properties of > tridiagonal matrices? It is not a question of matrix structure (tridiagonal, sparse) but the eigenvalue spectrum. For your matrix, there are large number of eigenvalues clustered to -1 which is the spectrum bottom, and this is bad for Krylov methods. > What other scipy routine can I use in order to > diagonalize my matrix in an efficient way? scipy.linalg.eig_banded From ralf.gommers at gmail.com Wed Sep 27 17:41:35 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Thu, 28 Sep 2017 10:41:35 +1300 Subject: [SciPy-User] ANN: first SciPy 1.0.0 release candidate Message-ID: Hi all, I'm excited to be able to announce the availability of the first release candidate of Scipy 1.0. This is a big release, and a version number that has been 16 years in the making. It contains a few more deprecations and backwards incompatible changes than an average release. Therefore please do test it on your own code, and report any issues on the Github issue tracker or on the scipy-dev mailing list. Sources and binary wheels can be found at https://pypi.python.org/pypi/scipy and https://github.com/scipy/scipy/releases/tag/v1.0.0rc1. To install with pip: pip install --pre --upgrade scipy Thanks to everyone who contributed to this release! Ralf Pull requests merged after v1.0.0b1: - `#7876 `__: GEN: Add comments to the tests for clarification - `#7891 `__: ENH: backport #7879 to 1.0.x - `#7902 `__: MAINT: signal: Make freqz handling of multidim. arrays match... - `#7905 `__: REV: restore wminkowski - `#7908 `__: FIX: Avoid bad ``__del__`` (close) behavior - `#7918 `__: TST: mark two optimize.linprog tests as xfail. See gh-7877. - `#7929 `__: MAINT: changed defaults to lower in sytf2, sytrf and hetrf - `#7938 `__: MAINT: backports from 1.0.x - `#7939 `__: Fix umfpack solver construction for win-amd64 ========================== SciPy 1.0.0 Release Notes ========================== .. note:: Scipy 1.0.0 is not released yet! .. contents:: SciPy 1.0.0 is the culmination of 8 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Moreover, our development attention will now shift to bug-fix releases on the 1.0.x branch, and on adding new features on the master branch. Some of the highlights of this release are: - Major build improvements. Windows wheels are available on PyPI for the first time, and continuous integration has been set up on Windows and OS X in addition to Linux. - A set of new ODE solvers and a unified interface to them (`scipy.integrate.solve_ivp`). - Two new trust region optimizers and a new linear programming method, with improved performance compared to what `scipy.optimize` offered previously. - Many new BLAS and LAPACK functions were wrapped. The BLAS wrappers are now complete. This release requires Python 2.7 or 3.4+ and NumPy 1.8.2 or greater. This is also the last release to support LAPACK 3.1.x - 3.3.x. Moving the lowest supported LAPACK version to >3.2.x was long blocked by Apple Accelerate providing the LAPACK 3.2.1 API. We have decided that it's time to either drop Accelerate or, if there is enough interest, provide shims for functions added in more recent LAPACK versions so it can still be used. New features ============ `scipy.cluster` improvements ---------------------------- `scipy.cluster.hierarchy.optimal_leaf_ordering`, a function to reorder a linkage matrix to minimize distances between adjacent leaves, was added. `scipy.fftpack` improvements ---------------------------- N-dimensional versions of the discrete sine and cosine transforms and their inverses were added as ``dctn``, ``idctn``, ``dstn`` and ``idstn``. `scipy.integrate` improvements ------------------------------ A set of new ODE solvers have been added to `scipy.integrate`. The convenience function `scipy.integrate.solve_ivp` allows uniform access to all solvers. The individual solvers (``RK23``, ``RK45``, ``Radau``, ``BDF`` and ``LSODA``) can also be used directly. `scipy.linalg` improvements ---------------------------- The BLAS wrappers in `scipy.linalg.blas` have been completed. Added functions are ``*gbmv``, ``*hbmv``, ``*hpmv``, ``*hpr``, ``*hpr2``, ``*spmv``, ``*spr``, ``*tbmv``, ``*tbsv``, ``*tpmv``, ``*tpsv``, ``*trsm``, ``*trsv``, ``*sbmv``, ``*spr2``, Wrappers for the LAPACK functions ``*gels``, ``*stev``, ``*sytrd``, ``*hetrd``, ``*sytf2``, ``*hetrf``, ``*sytrf``, ``*sycon``, ``*hecon``, ``*gglse``, ``*stebz``, ``*stemr``, ``*sterf``, and ``*stein`` have been added. The function `scipy.linalg.subspace_angles` has been added to compute the subspace angles between two matrices. The function `scipy.linalg.clarkson_woodruff_transform` has been added. It finds low-rank matrix approximation via the Clarkson-Woodruff Transform. The functions `scipy.linalg.eigh_tridiagonal` and `scipy.linalg.eigvalsh_tridiagonal`, which find the eigenvalues and eigenvectors of tridiagonal hermitian/symmetric matrices, were added. `scipy.ndimage` improvements ---------------------------- Support for homogeneous coordinate transforms has been added to `scipy.ndimage.affine_transform`. The ``ndimage`` C code underwent a significant refactoring, and is now a lot easier to understand and maintain. `scipy.optimize` improvements ----------------------------- The methods ``trust-region-exact`` and ``trust-krylov`` have been added to the function `scipy.optimize.minimize`. These new trust-region methods solve the subproblem with higher accuracy at the cost of more Hessian factorizations (compared to dogleg) or more matrix vector products (compared to ncg) but usually require less nonlinear iterations and are able to deal with indefinite Hessians. They seem very competitive against the other Newton methods implemented in scipy. `scipy.optimize.linprog` gained an interior point method. Its performance is superior (both in accuracy and speed) to the older simplex method. `scipy.signal` improvements --------------------------- An argument ``fs`` (sampling frequency) was added to the following functions: ``firwin``, ``firwin2``, ``firls``, and ``remez``. This makes these functions consistent with many other functions in `scipy.signal` in which the sampling frequency can be specified. `scipy.signal.freqz` has been sped up significantly for FIR filters. `scipy.sparse` improvements --------------------------- Iterating over and slicing of CSC and CSR matrices is now faster by up to ~35%. The ``tocsr`` method of COO matrices is now several times faster. The ``diagonal`` method of sparse matrices now takes a parameter, indicating which diagonal to return. `scipy.sparse.linalg` improvements ---------------------------------- A new iterative solver for large-scale nonsymmetric sparse linear systems, `scipy.sparse.linalg.gcrotmk`, was added. It implements ``GCROT(m,k)``, a flexible variant of ``GCROT``. `scipy.sparse.linalg.lsmr` now accepts an initial guess, yielding potentially faster convergence. SuperLU was updated to version 5.2.1. `scipy.spatial` improvements ---------------------------- Many distance metrics in `scipy.spatial.distance` gained support for weights. The signatures of `scipy.spatial.distance.pdist` and `scipy.spatial.distance.cdist` were changed to ``*args, **kwargs`` in order to support a wider range of metrics (e.g. string-based metrics that need extra keywords). Also, an optional ``out`` parameter was added to ``pdist`` and ``cdist`` allowing the user to specify where the resulting distance matrix is to be stored `scipy.stats` improvements -------------------------- The methods ``cdf`` and ``logcdf`` were added to `scipy.stats.multivariate_normal`, providing the cumulative distribution function of the multivariate normal distribution. New statistical distance functions were added, namely `scipy.stats.wasserstein_distance` for the first Wasserstein distance and `scipy.stats.energy_distance` for the energy distance. Deprecated features =================== The following functions in `scipy.misc` are deprecated: ``bytescale``, ``fromimage``, ``imfilter``, ``imread``, ``imresize``, ``imrotate``, ``imsave``, ``imshow`` and ``toimage``. Most of those functions have unexpected behavior (like rescaling and type casting image data without the user asking for that). Other functions simply have better alternatives. ``scipy.interpolate.interpolate_wrapper`` and all functions in that submodule are deprecated. This was a never finished set of wrapper functions which is not relevant anymore. The ``fillvalue`` of `scipy.signal.convolve2d` will be cast directly to the dtypes of the input arrays in the future and checked that it is a scalar or an array with a single element. ``scipy.spatial.distance.matching`` is deprecated. It is an alias of `scipy.spatial.distance.hamming`, which should be used instead. Implementation of `scipy.spatial.distance.wminkowski` was based on a wrong interpretation of the metric definition. In scipy 1.0 it has been just deprecated in the documentation to keep retro-compatibility but is recommended to use the new version of `scipy.spatial.distance.minkowski` that implements the correct behaviour. Positional arguments of `scipy.spatial.distance.pdist` and `scipy.spatial.distance.cdist` should be replaced with their keyword version. Backwards incompatible changes ============================== The following deprecated functions have been removed from `scipy.stats`: ``betai``, ``chisqprob``, ``f_value``, ``histogram``, ``histogram2``, ``pdf_fromgamma``, ``signaltonoise``, ``square_of_sums``, ``ss`` and ``threshold``. The following deprecated functions have been removed from `scipy.stats.mstats`: ``betai``, ``f_value_wilks_lambda``, ``signaltonoise`` and ``threshold``. The deprecated ``a`` and ``reta`` keywords have been removed from `scipy.stats.shapiro`. The deprecated functions ``sparse.csgraph.cs_graph_components`` and ``sparse.linalg.symeig`` have been removed from `scipy.sparse`. The following deprecated keywords have been removed in `scipy.sparse.linalg`: ``drop_tol`` from ``splu``, and ``xtype`` from ``bicg``, ``bicgstab``, ``cg``, ``cgs``, ``gmres``, ``qmr`` and ``minres``. The deprecated functions ``expm2`` and ``expm3`` have been removed from `scipy.linalg`. The deprecated keyword ``q`` was removed from `scipy.linalg.expm`. And the deprecated submodule ``linalg.calc_lwork`` was removed. The deprecated functions ``C2K``, ``K2C``, ``F2C``, ``C2F``, ``F2K`` and ``K2F`` have been removed from `scipy.constants`. The deprecated ``ppform`` class was removed from `scipy.interpolate`. The deprecated keyword ``iprint`` was removed from `scipy.optimize.fmin_cobyla`. The default value for the ``zero_phase`` keyword of `scipy.signal.decimate` has been changed to True. The ``kmeans`` and ``kmeans2`` functions in `scipy.cluster.vq` changed the method used for random initialization, so using a fixed random seed will not necessarily produce the same results as in previous versions. `scipy.special.gammaln` does not accept complex arguments anymore. The deprecated functions ``sph_jn``, ``sph_yn``, ``sph_jnyn``, ``sph_in``, ``sph_kn``, and ``sph_inkn`` have been removed. Users should instead use the functions ``spherical_jn``, ``spherical_yn``, ``spherical_in``, and ``spherical_kn``. Be aware that the new functions have different signatures. The cross-class properties of `scipy.signal.lti` systems have been removed. The following properties/setters have been removed: Name - (accessing/setting has been removed) - (setting has been removed) * StateSpace - (``num``, ``den``, ``gain``) - (``zeros``, ``poles``) * TransferFunction (``A``, ``B``, ``C``, ``D``, ``gain``) - (``zeros``, ``poles``) * ZerosPolesGain (``A``, ``B``, ``C``, ``D``, ``num``, ``den``) - () ``signal.freqz(b, a)`` with ``b`` or ``a`` >1-D raises a ``ValueError``. This was a corner case for which it was unclear that the behavior was well-defined. The method ``var`` of `scipy.stats.dirichlet` now returns a scalar rather than an ndarray when the length of alpha is 1. Other changes ============= SciPy now has a formal governance structure. It consists of a BDFL (Pauli Virtanen) and a Steering Committee. See `the governance document < https://github.com/scipy/scipy/blob/master/doc/source/dev/governance/governance.rst >`_ for details. It is now possible to build SciPy on Windows with MSVC + gfortran! Continuous integration has been set up for this build configuration on Appveyor, building against OpenBLAS. Continuous integration for OS X has been set up on TravisCI. The SciPy test suite has been migrated from ``nose`` to ``pytest``. ``scipy/_distributor_init.py`` was added to allow redistributors of SciPy to add custom code that needs to run when importing SciPy (e.g. checks for hardware, DLL search paths, etc.). Support for PEP 518 (specifying build system requirements) was added - see ``pyproject.toml`` in the root of the SciPy repository. In order to have consistent function names, the function ``scipy.linalg.solve_lyapunov`` is renamed to `scipy.linalg.solve_continuous_lyapunov`. The old name is kept for backwards-compatibility. -------------- next part -------------- An HTML attachment was scrubbed... URL: From andy.terrel at gmail.com Wed Sep 27 22:31:28 2017 From: andy.terrel at gmail.com (Andy Ray Terrel) Date: Wed, 27 Sep 2017 21:31:28 -0500 Subject: [SciPy-User] ANN: first SciPy 1.0.0 release candidate In-Reply-To: References: Message-ID: This is amazing! Thanks for all the hard work everyone. On Wed, Sep 27, 2017 at 4:41 PM, Ralf Gommers wrote: > Hi all, > > I'm excited to be able to announce the availability of the first release > candidate of Scipy 1.0. This is a big release, and a version number that > has been 16 years in the making. It contains a few more deprecations and > backwards incompatible changes than an average release. Therefore please do > test it on your own code, and report any issues on the Github issue tracker > or on the scipy-dev mailing list. > > Sources and binary wheels can be found at https://pypi.python.org/pypi/ > scipy and https://github.com/scipy/scipy/releases/tag/v1.0.0rc1. To > install with pip: > > pip install --pre --upgrade scipy > > Thanks to everyone who contributed to this release! > > Ralf > > > > Pull requests merged after v1.0.0b1: > > - `#7876 `__: GEN: Add comments > to the tests for clarification > - `#7891 `__: ENH: backport > #7879 to 1.0.x > - `#7902 `__: MAINT: signal: > Make freqz handling of multidim. arrays match... > - `#7905 `__: REV: restore > wminkowski > - `#7908 `__: FIX: Avoid bad > ``__del__`` (close) behavior > - `#7918 `__: TST: mark two > optimize.linprog tests as xfail. See gh-7877. > - `#7929 `__: MAINT: changed > defaults to lower in sytf2, sytrf and hetrf > - `#7938 `__: MAINT: backports > from 1.0.x > - `#7939 `__: Fix umfpack > solver construction for win-amd64 > > > > > ========================== > SciPy 1.0.0 Release Notes > ========================== > > .. note:: Scipy 1.0.0 is not released yet! > > .. contents:: > > SciPy 1.0.0 is the culmination of 8 months of hard work. It contains > many new features, numerous bug-fixes, improved test coverage and > better documentation. There have been a number of deprecations and > API changes in this release, which are documented below. All users > are encouraged to upgrade to this release, as there are a large number > of bug-fixes and optimizations. Moreover, our development attention > will now shift to bug-fix releases on the 1.0.x branch, and on adding > new features on the master branch. > > Some of the highlights of this release are: > > - Major build improvements. Windows wheels are available on PyPI for the > first time, and continuous integration has been set up on Windows and OS > X > in addition to Linux. > - A set of new ODE solvers and a unified interface to them > (`scipy.integrate.solve_ivp`). > - Two new trust region optimizers and a new linear programming method, with > improved performance compared to what `scipy.optimize` offered > previously. > - Many new BLAS and LAPACK functions were wrapped. The BLAS wrappers are > now > complete. > > This release requires Python 2.7 or 3.4+ and NumPy 1.8.2 or greater. > > This is also the last release to support LAPACK 3.1.x - 3.3.x. Moving the > lowest supported LAPACK version to >3.2.x was long blocked by Apple > Accelerate > providing the LAPACK 3.2.1 API. We have decided that it's time to either > drop > Accelerate or, if there is enough interest, provide shims for functions > added > in more recent LAPACK versions so it can still be used. > > > New features > ============ > > `scipy.cluster` improvements > ---------------------------- > > `scipy.cluster.hierarchy.optimal_leaf_ordering`, a function to reorder a > linkage matrix to minimize distances between adjacent leaves, was added. > > > `scipy.fftpack` improvements > ---------------------------- > > N-dimensional versions of the discrete sine and cosine transforms and their > inverses were added as ``dctn``, ``idctn``, ``dstn`` and ``idstn``. > > > `scipy.integrate` improvements > ------------------------------ > > A set of new ODE solvers have been added to `scipy.integrate`. The > convenience > function `scipy.integrate.solve_ivp` allows uniform access to all solvers. > The individual solvers (``RK23``, ``RK45``, ``Radau``, ``BDF`` and > ``LSODA``) > can also be used directly. > > > `scipy.linalg` improvements > ---------------------------- > > The BLAS wrappers in `scipy.linalg.blas` have been completed. Added > functions > are ``*gbmv``, ``*hbmv``, ``*hpmv``, ``*hpr``, ``*hpr2``, ``*spmv``, > ``*spr``, > ``*tbmv``, ``*tbsv``, ``*tpmv``, ``*tpsv``, ``*trsm``, ``*trsv``, > ``*sbmv``, > ``*spr2``, > > Wrappers for the LAPACK functions ``*gels``, ``*stev``, ``*sytrd``, > ``*hetrd``, > ``*sytf2``, ``*hetrf``, ``*sytrf``, ``*sycon``, ``*hecon``, ``*gglse``, > ``*stebz``, ``*stemr``, ``*sterf``, and ``*stein`` have been added. > > The function `scipy.linalg.subspace_angles` has been added to compute the > subspace angles between two matrices. > > The function `scipy.linalg.clarkson_woodruff_transform` has been added. > It finds low-rank matrix approximation via the Clarkson-Woodruff Transform. > > The functions `scipy.linalg.eigh_tridiagonal` and > `scipy.linalg.eigvalsh_tridiagonal`, which find the eigenvalues and > eigenvectors of tridiagonal hermitian/symmetric matrices, were added. > > > `scipy.ndimage` improvements > ---------------------------- > > Support for homogeneous coordinate transforms has been added to > `scipy.ndimage.affine_transform`. > > The ``ndimage`` C code underwent a significant refactoring, and is now > a lot easier to understand and maintain. > > > `scipy.optimize` improvements > ----------------------------- > > The methods ``trust-region-exact`` and ``trust-krylov`` have been added to > the > function `scipy.optimize.minimize`. These new trust-region methods solve > the > subproblem with higher accuracy at the cost of more Hessian factorizations > (compared to dogleg) or more matrix vector products (compared to ncg) but > usually require less nonlinear iterations and are able to deal with > indefinite > Hessians. They seem very competitive against the other Newton methods > implemented in scipy. > > `scipy.optimize.linprog` gained an interior point method. Its performance > is > superior (both in accuracy and speed) to the older simplex method. > > > `scipy.signal` improvements > --------------------------- > > An argument ``fs`` (sampling frequency) was added to the following > functions: > ``firwin``, ``firwin2``, ``firls``, and ``remez``. This makes these > functions > consistent with many other functions in `scipy.signal` in which the > sampling > frequency can be specified. > > `scipy.signal.freqz` has been sped up significantly for FIR filters. > > > `scipy.sparse` improvements > --------------------------- > > Iterating over and slicing of CSC and CSR matrices is now faster by up to > ~35%. > > The ``tocsr`` method of COO matrices is now several times faster. > > The ``diagonal`` method of sparse matrices now takes a parameter, > indicating > which diagonal to return. > > > `scipy.sparse.linalg` improvements > ---------------------------------- > > A new iterative solver for large-scale nonsymmetric sparse linear systems, > `scipy.sparse.linalg.gcrotmk`, was added. It implements ``GCROT(m,k)``, a > flexible variant of ``GCROT``. > > `scipy.sparse.linalg.lsmr` now accepts an initial guess, yielding > potentially > faster convergence. > > SuperLU was updated to version 5.2.1. > > > `scipy.spatial` improvements > ---------------------------- > > Many distance metrics in `scipy.spatial.distance` gained support for > weights. > > The signatures of `scipy.spatial.distance.pdist` and > `scipy.spatial.distance.cdist` were changed to ``*args, **kwargs`` in > order to > support a wider range of metrics (e.g. string-based metrics that need extra > keywords). Also, an optional ``out`` parameter was added to ``pdist`` and > ``cdist`` allowing the user to specify where the resulting distance matrix > is > to be stored > > > `scipy.stats` improvements > -------------------------- > > The methods ``cdf`` and ``logcdf`` were added to > `scipy.stats.multivariate_normal`, providing the cumulative distribution > function of the multivariate normal distribution. > > New statistical distance functions were added, namely > `scipy.stats.wasserstein_distance` for the first Wasserstein distance and > `scipy.stats.energy_distance` for the energy distance. > > > Deprecated features > =================== > > The following functions in `scipy.misc` are deprecated: ``bytescale``, > ``fromimage``, ``imfilter``, ``imread``, ``imresize``, ``imrotate``, > ``imsave``, ``imshow`` and ``toimage``. Most of those functions have > unexpected > behavior (like rescaling and type casting image data without the user > asking > for that). Other functions simply have better alternatives. > > ``scipy.interpolate.interpolate_wrapper`` and all functions in that > submodule > are deprecated. This was a never finished set of wrapper functions which > is > not relevant anymore. > > The ``fillvalue`` of `scipy.signal.convolve2d` will be cast directly to the > dtypes of the input arrays in the future and checked that it is a scalar or > an array with a single element. > > ``scipy.spatial.distance.matching`` is deprecated. It is an alias of > `scipy.spatial.distance.hamming`, which should be used instead. > > Implementation of `scipy.spatial.distance.wminkowski` was based on a wrong > interpretation of the metric definition. In scipy 1.0 it has been just > deprecated in the documentation to keep retro-compatibility but is > recommended > to use the new version of `scipy.spatial.distance.minkowski` that > implements > the correct behaviour. > > Positional arguments of `scipy.spatial.distance.pdist` and > `scipy.spatial.distance.cdist` should be replaced with their keyword > version. > > > Backwards incompatible changes > ============================== > > The following deprecated functions have been removed from `scipy.stats`: > ``betai``, ``chisqprob``, ``f_value``, ``histogram``, ``histogram2``, > ``pdf_fromgamma``, ``signaltonoise``, ``square_of_sums``, ``ss`` and > ``threshold``. > > The following deprecated functions have been removed from > `scipy.stats.mstats`: > ``betai``, ``f_value_wilks_lambda``, ``signaltonoise`` and ``threshold``. > > The deprecated ``a`` and ``reta`` keywords have been removed from > `scipy.stats.shapiro`. > > The deprecated functions ``sparse.csgraph.cs_graph_components`` and > ``sparse.linalg.symeig`` have been removed from `scipy.sparse`. > > The following deprecated keywords have been removed in > `scipy.sparse.linalg`: > ``drop_tol`` from ``splu``, and ``xtype`` from ``bicg``, ``bicgstab``, > ``cg``, > ``cgs``, ``gmres``, ``qmr`` and ``minres``. > > The deprecated functions ``expm2`` and ``expm3`` have been removed from > `scipy.linalg`. The deprecated keyword ``q`` was removed from > `scipy.linalg.expm`. And the deprecated submodule ``linalg.calc_lwork`` > was > removed. > > The deprecated functions ``C2K``, ``K2C``, ``F2C``, ``C2F``, ``F2K`` and > ``K2F`` have been removed from `scipy.constants`. > > The deprecated ``ppform`` class was removed from `scipy.interpolate`. > > The deprecated keyword ``iprint`` was removed from > `scipy.optimize.fmin_cobyla`. > > The default value for the ``zero_phase`` keyword of `scipy.signal.decimate` > has been changed to True. > > The ``kmeans`` and ``kmeans2`` functions in `scipy.cluster.vq` changed the > method used for random initialization, so using a fixed random seed will > not necessarily produce the same results as in previous versions. > > `scipy.special.gammaln` does not accept complex arguments anymore. > > The deprecated functions ``sph_jn``, ``sph_yn``, ``sph_jnyn``, ``sph_in``, > ``sph_kn``, and ``sph_inkn`` have been removed. Users should instead use > the functions ``spherical_jn``, ``spherical_yn``, ``spherical_in``, and > ``spherical_kn``. Be aware that the new functions have different > signatures. > > The cross-class properties of `scipy.signal.lti` systems have been removed. > The following properties/setters have been removed: > > Name - (accessing/setting has been removed) - (setting has been removed) > > * StateSpace - (``num``, ``den``, ``gain``) - (``zeros``, ``poles``) > * TransferFunction (``A``, ``B``, ``C``, ``D``, ``gain``) - (``zeros``, > ``poles``) > * ZerosPolesGain (``A``, ``B``, ``C``, ``D``, ``num``, ``den``) - () > > ``signal.freqz(b, a)`` with ``b`` or ``a`` >1-D raises a ``ValueError``. > This > was a corner case for which it was unclear that the behavior was > well-defined. > > The method ``var`` of `scipy.stats.dirichlet` now returns a scalar rather > than > an ndarray when the length of alpha is 1. > > > Other changes > ============= > > SciPy now has a formal governance structure. It consists of a BDFL (Pauli > Virtanen) and a Steering Committee. See `the governance document > dev/governance/governance.rst>`_ > for details. > > It is now possible to build SciPy on Windows with MSVC + gfortran! > Continuous > integration has been set up for this build configuration on Appveyor, > building > against OpenBLAS. > > Continuous integration for OS X has been set up on TravisCI. > > The SciPy test suite has been migrated from ``nose`` to ``pytest``. > > ``scipy/_distributor_init.py`` was added to allow redistributors of SciPy > to > add custom code that needs to run when importing SciPy (e.g. checks for > hardware, DLL search paths, etc.). > > Support for PEP 518 (specifying build system requirements) was added - see > ``pyproject.toml`` in the root of the SciPy repository. > > In order to have consistent function names, the function > ``scipy.linalg.solve_lyapunov`` is renamed to > `scipy.linalg.solve_continuous_lyapunov`. The old name is kept for > backwards-compatibility. > > > > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Thu Sep 28 00:10:57 2017 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 27 Sep 2017 22:10:57 -0600 Subject: [SciPy-User] NumPy 1.13.2 released. Message-ID: HI All, On behalf of the NumPy team, I am pleased to annouce the release of Numpy 1.13.2. This is a bugfix release for some problems found since 1.13.1. The most important fixes are for CVE-2017-12852 and temporary elision. Users of earlier versions of 1.13 should upgrade. The Python versions supported are 2.7 and 3.4 - 3.6. The Python 3.6 wheels available from PIP are built with Python 3.6.2 and should be compatible with all previous versions of Python 3.6. The Windows wheels are now built with OpenBlas instead ATLAS, which should improve the performance of the linearalgebra functions. Contributors ============ A total of 12 people contributed to this release. People with a "+" by their names contributed a patch for the first time. * Allan Haldane * Brandon Carter * Charles Harris * Eric Wieser * Iryna Shcherbina + * James Bourbeau + * Jonathan Helmus * Julian Taylor * Matti Picus * Michael Lamparski + * Michael Seifert * Ralf Gommers Pull requests merged ==================== A total of 20 pull requests were merged for this release. * #9390 BUG: Return the poly1d coefficients array directly * #9555 BUG: Fix regression in 1.13.x in distutils.mingw32ccompiler. * #9556 BUG: Fix true_divide when dtype=np.float64 specified. * #9557 DOC: Fix some rst markup in numpy/doc/basics.py. * #9558 BLD: Remove -xhost flag from IntelFCompiler. * #9559 DOC: Removes broken docstring example (source code, png, pdf)... * #9580 BUG: Add hypot and cabs functions to WIN32 blacklist. * #9732 BUG: Make scalar function elision check if temp is writeable. * #9736 BUG: Various fixes to np.gradient * #9742 BUG: Fix np.pad for CVE-2017-12852 * #9744 BUG: Check for exception in sort functions, add tests * #9745 DOC: Add whitespace after "versionadded::" directive so it actually... * #9746 BUG: Memory leak in np.dot of size 0 * #9747 BUG: Adjust gfortran version search regex * #9757 BUG: Cython 0.27 breaks NumPy on Python 3. * #9764 BUG: Ensure `_npy_scaled_cexp{,f,l}` is defined when needed. * #9765 BUG: PyArray_CountNonzero does not check for exceptions * #9766 BUG: Fixes histogram monotonicity check for unsigned bin values * #9767 BUG: Ensure consistent result dtype of count_nonzero * #9771 BUG, MAINT: Fix mtrand for Cython 0.27. Enjoy Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Thu Sep 28 04:48:41 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Thu, 28 Sep 2017 21:48:41 +1300 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On Mon, Sep 18, 2017 at 8:58 PM, Ralf Gommers wrote: > > > On Mon, Sep 18, 2017 at 1:16 AM, Joe Harrington > wrote: > >> On Sun, 17 Sep 2017 22:48:35 +1200 Ralf Gommers >> wrote: >> >> > I'm excited to be able to announce the availability of the first beta >> > release of Scipy 1.0. >> > >> [I won't quote the entire release notes!] >> >> Awesome news, looooong in the making! >> >> The number of deprecations and removals is understandably large. >> I think we need a centralized resource for navigating deprecations, >> removals, and changes. It could be a simple web page, or a more >> involved database. It should list the change, give or link to the >> rationale, and state the date and package version number. Most >> importantly, it should give the suggested replacement(s). >> >> Googling "scipy deprecation" found only this page, plus a bunch of >> release notes and discussions. The page was last updated on Jan 16, >> 2017, and is limited to C-API deprecations: >> >> https://docs.scipy.org/doc/numpy-dev/reference/c-api.deprecations.html >> >> Even once such a document/page/searchable database exists, many will be >> unaware of it, or not know where to find it, so referring to it at the >> top of any list of deprecations/removals would be good. >> >> Once it exists, deprecations could be entered before being implemented, >> and deprecation warnings could refer to an index number in this >> resource, where the user could go for more information on how to fix the >> issue. >> > > Thanks for your thoughts Joe! I'm not sure a database is the way to go, > but you make a good point. We have to guide users a bit more in what to do > about deprecations. I think clear instructions about upgrading in both the > html docs and release announcements would be useful. I don't think there's > many users who run ``python -Wd my_code.py`` .(making depreacations that > affect your code visible) .... > Done in https://github.com/scipy/scipy/pull/7947 Ralf -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjol at tjol.eu Thu Sep 28 05:53:24 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Thu, 28 Sep 2017 11:53:24 +0200 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On 2017-09-18 10:58, Ralf Gommers wrote: > I don't think there's many users who run ``python -Wd my_code.py``. > (making depreacations that affect your code visible) .... I'm sure they don't, especially seeing as it's not obvious from the Python docs that "-Wd" does *anything*: requesting the default behaviour *should* be a noop... However, the default for IPython is to display DeprecationsWarning and PendingDeprecationWarning, so quite a lot of people will actually see them. -- Thomas Jollans From ndbecker2 at gmail.com Thu Sep 28 07:07:21 2017 From: ndbecker2 at gmail.com (Neal Becker) Date: Thu, 28 Sep 2017 11:07:21 +0000 Subject: [SciPy-User] order_filter Message-ID: Is there a description of what is "order_filter"? https://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.order_filter.html#scipy.signal.order_filter Googling for it is almost impossible (unless you want to order filters from amazon) -------------- next part -------------- An HTML attachment was scrubbed... URL: From jh at physics.ucf.edu Thu Sep 28 09:01:02 2017 From: jh at physics.ucf.edu (Joe Harrington) Date: Thu, 28 Sep 2017 09:01:02 -0400 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: (message from Ralf Gommers on Thu, 28 Sep 2017 21:48:41 +1300) Message-ID: Um... This commit has to do with wheels, not deprecations. Great that we're flying with wheels up, though. :-) --jh-- On Thu, 28 Sep 2017 21:48:41 +1300, Ralf Gommers wrote: > Done in https://github.com/scipy/scipy/pull/7947 >> On Mon, Sep 18, 2017 at 1:16 AM, Joe Harrington >> wrote: >> >>> On Sun, 17 Sep 2017 22:48:35 +1200 Ralf Gommers >>> wrote: >>> >>> > I'm excited to be able to announce the availability of the first beta >>> > release of Scipy 1.0. >>> > >>> [I won't quote the entire release notes!] >>> >>> Awesome news, looooong in the making! >>> >>> The number of deprecations and removals is understandably large. >>> I think we need a centralized resource for navigating deprecations, >>> removals, and changes. It could be a simple web page, or a more >>> involved database. It should list the change, give or link to the >>> rationale, and state the date and package version number. Most >>> importantly, it should give the suggested replacement(s). >>> >>> Googling "scipy deprecation" found only this page, plus a bunch of >>> release notes and discussions. The page was last updated on Jan 16, >>> 2017, and is limited to C-API deprecations: >>> >>> https://docs.scipy.org/doc/numpy-dev/reference/c-api.deprecations.html >>> >>> Even once such a document/page/searchable database exists, many will be >>> unaware of it, or not know where to find it, so referring to it at the >>> top of any list of deprecations/removals would be good. >>> >>> Once it exists, deprecations could be entered before being implemented, >>> and deprecation warnings could refer to an index number in this >>> resource, where the user could go for more information on how to fix the >>> issue. >>> >> >> Thanks for your thoughts Joe! I'm not sure a database is the way to go, >> but you make a good point. We have to guide users a bit more in what to do >> about deprecations. I think clear instructions about upgrading in both the >> html docs and release announcements would be useful. I don't think there's >> many users who run ``python -Wd my_code.py`` .(making depreacations that >> affect your code visible) .... >> > From jh at physics.ucf.edu Thu Sep 28 09:44:13 2017 From: jh at physics.ucf.edu (Joe Harrington) Date: Thu, 28 Sep 2017 09:44:13 -0400 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: (message from Joe Harrington on Thu, 28 Sep 2017 09:01:02 -0400) Message-ID: Ok, I saw the -Wd at the end, but it just requests the default, so how does this change anything? If it's just an ipython benefit, say that it's recommended for ipython users. Also, remember that for the vast majority of users, they don't install a new version, they suddenly discover that their system now has one, through some software update. Think students and casual users, as opposed to PhD research scientists experienced in software management. But, what we really need is more information about what to DO about the deprecations. Not, "we pulled the rug out from under you", but "so that we don't pull the rug out from under you, do this". And, in a more-obvious place than the release notes, which most users don't ever see. --jh-- >Um... This commit has to do with wheels, not deprecations. Great that >we're flying with wheels up, though. :-) > >--jh-- > >On Thu, 28 Sep 2017 21:48:41 +1300, Ralf Gommers >wrote: > >> Done in https://github.com/scipy/scipy/pull/7947 > >>> On Mon, Sep 18, 2017 at 1:16 AM, Joe Harrington >>> wrote: >>> >>>> On Sun, 17 Sep 2017 22:48:35 +1200 Ralf Gommers >>>> wrote: >>>> >>>> > I'm excited to be able to announce the availability of the first beta >>>> > release of Scipy 1.0. >>>> > >>>> [I won't quote the entire release notes!] >>>> >>>> Awesome news, looooong in the making! >>>> >>>> The number of deprecations and removals is understandably large. >>>> I think we need a centralized resource for navigating deprecations, >>>> removals, and changes. It could be a simple web page, or a more >>>> involved database. It should list the change, give or link to the >>>> rationale, and state the date and package version number. Most >>>> importantly, it should give the suggested replacement(s). >>>> >>>> Googling "scipy deprecation" found only this page, plus a bunch of >>>> release notes and discussions. The page was last updated on Jan 16, >>>> 2017, and is limited to C-API deprecations: >>>> >>>> https://docs.scipy.org/doc/numpy-dev/reference/c-api.deprecations.html >>>> >>>> Even once such a document/page/searchable database exists, many will be >>>> unaware of it, or not know where to find it, so referring to it at the >>>> top of any list of deprecations/removals would be good. >>>> >>>> Once it exists, deprecations could be entered before being implemented, >>>> and deprecation warnings could refer to an index number in this >>>> resource, where the user could go for more information on how to fix the >>>> issue. >>>> >>> >>> Thanks for your thoughts Joe! I'm not sure a database is the way to go, >>> but you make a good point. We have to guide users a bit more in what to do >>> about deprecations. I think clear instructions about upgrading in both the >>> html docs and release announcements would be useful. I don't think there's >>> many users who run ``python -Wd my_code.py`` .(making depreacations that >>> affect your code visible) .... >>> >> > > From warren.weckesser at gmail.com Thu Sep 28 11:21:50 2017 From: warren.weckesser at gmail.com (Warren Weckesser) Date: Thu, 28 Sep 2017 11:21:50 -0400 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On Thu, Sep 28, 2017 at 9:44 AM, Joe Harrington wrote: > Ok, I saw the -Wd at the end, but it just requests the default, so how > does this change anything? Here's a fun sentence from the documentation at https://docs.python.org/3.6/library/warnings.html#updating-code-for-new-versions-of-python regarding the option -Wd: > This enables default handling for all warnings, including those that are ignored by default. Fortunately, the sentences previous to that one explain that when -Wd is used, DeprecationWarnings (usually ignored) are made visible. Warren If it's just an ipython benefit, say that > it's recommended for ipython users. > > Also, remember that for the vast majority of users, they don't install a > new version, they suddenly discover that their system now has one, > through some software update. Think students and casual users, as > opposed to PhD research scientists experienced in software management. > > But, what we really need is more information about what to DO about the > deprecations. Not, "we pulled the rug out from under you", but "so that > we don't pull the rug out from under you, do this". And, in a > more-obvious place than the release notes, which most users don't ever > see. > > --jh-- > > >Um... This commit has to do with wheels, not deprecations. Great that > >we're flying with wheels up, though. :-) > > > >--jh-- > > > >On Thu, 28 Sep 2017 21:48:41 +1300, Ralf Gommers > > >wrote: > > > >> Done in https://github.com/scipy/scipy/pull/7947 > > > >>> On Mon, Sep 18, 2017 at 1:16 AM, Joe Harrington > >>> wrote: > >>> > >>>> On Sun, 17 Sep 2017 22:48:35 +1200 Ralf Gommers < > ralf.gommers at gmail.com> > >>>> wrote: > >>>> > >>>> > I'm excited to be able to announce the availability of the first > beta > >>>> > release of Scipy 1.0. > >>>> > > >>>> [I won't quote the entire release notes!] > >>>> > >>>> Awesome news, looooong in the making! > >>>> > >>>> The number of deprecations and removals is understandably large. > >>>> I think we need a centralized resource for navigating deprecations, > >>>> removals, and changes. It could be a simple web page, or a more > >>>> involved database. It should list the change, give or link to the > >>>> rationale, and state the date and package version number. Most > >>>> importantly, it should give the suggested replacement(s). > >>>> > >>>> Googling "scipy deprecation" found only this page, plus a bunch of > >>>> release notes and discussions. The page was last updated on Jan 16, > >>>> 2017, and is limited to C-API deprecations: > >>>> > >>>> https://docs.scipy.org/doc/numpy-dev/reference/c-api. > deprecations.html > >>>> > >>>> Even once such a document/page/searchable database exists, many will > be > >>>> unaware of it, or not know where to find it, so referring to it at the > >>>> top of any list of deprecations/removals would be good. > >>>> > >>>> Once it exists, deprecations could be entered before being > implemented, > >>>> and deprecation warnings could refer to an index number in this > >>>> resource, where the user could go for more information on how to fix > the > >>>> issue. > >>>> > >>> > >>> Thanks for your thoughts Joe! I'm not sure a database is the way to go, > >>> but you make a good point. We have to guide users a bit more in what > to do > >>> about deprecations. I think clear instructions about upgrading in both > the > >>> html docs and release announcements would be useful. I don't think > there's > >>> many users who run ``python -Wd my_code.py`` .(making depreacations > that > >>> affect your code visible) .... > >>> > >> > > > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From warren.weckesser at gmail.com Thu Sep 28 12:02:26 2017 From: warren.weckesser at gmail.com (Warren Weckesser) Date: Thu, 28 Sep 2017 12:02:26 -0400 Subject: [SciPy-User] order_filter In-Reply-To: References: Message-ID: On Thu, Sep 28, 2017 at 7:07 AM, Neal Becker wrote: > Is there a description of what is "order_filter"? > https://docs.scipy.org/doc/scipy/reference/generated/ > scipy.signal.order_filter.html#scipy.signal.order_filter > > Googling for it is almost impossible (unless you want to order filters > from amazon) > > The word "order" is used in the sense of "order statistics" ( https://en.wikipedia.org/wiki/Order_statistic). The filter replaces each element with the `rank` element from the sorted list of values in its "neighborhood", where the neighborhood is determined by `domain`. The values of "neighbors" outside the array are taken to be 0 (apparently). It duplicates functionality provided by `scipy.ndimage.rank_filter` ( https://docs.scipy.org/doc/scipy/reference/generated/scipy.ndimage.rank_filter.html ). For example, In [27]: from scipy.signal import order_filter In [28]: from scipy.ndimage import rank_filter In [29]: np.random.seed(42) In [30]: x = np.random.randint(1, 100, size=(5, 8)) In [31]: x Out[31]: array([[52, 93, 15, 72, 61, 21, 83, 87], [75, 75, 88, 24, 3, 22, 53, 2], [88, 30, 38, 2, 64, 60, 21, 33], [76, 58, 22, 89, 49, 91, 59, 42], [92, 60, 80, 15, 62, 62, 47, 62]]) In [32]: order_filter(x, np.ones((3, 3)), 0) Out[32]: array([[ 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 15., 2., 2., 2., 3., 2., 0.], [ 0., 22., 2., 2., 2., 3., 2., 0.], [ 0., 22., 2., 2., 2., 21., 21., 0.], [ 0., 0., 0., 0., 0., 0., 0., 0.]]) In [33]: rank_filter(x, 0, size=3, mode='constant') Out[33]: array([[ 0, 0, 0, 0, 0, 0, 0, 0], [ 0, 15, 2, 2, 2, 3, 2, 0], [ 0, 22, 2, 2, 2, 3, 2, 0], [ 0, 22, 2, 2, 2, 21, 21, 0], [ 0, 0, 0, 0, 0, 0, 0, 0]]) In [34]: order_filter(x, np.ones((3, 3)), 1) Out[34]: array([[ 0., 0., 0., 0., 0., 0., 0., 0.], [ 0., 30., 15., 3., 3., 21., 21., 0.], [ 0., 30., 22., 3., 3., 21., 21., 0.], [ 0., 30., 15., 15., 15., 47., 33., 0.], [ 0., 0., 0., 0., 0., 0., 0., 0.]]) In [35]: rank_filter(x, 1, size=3, mode='constant') Out[35]: array([[ 0, 0, 0, 0, 0, 0, 0, 0], [ 0, 30, 15, 3, 3, 21, 21, 0], [ 0, 30, 22, 3, 3, 21, 21, 0], [ 0, 30, 15, 15, 15, 47, 33, 0], [ 0, 0, 0, 0, 0, 0, 0, 0]]) Warren _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jh at physics.ucf.edu Thu Sep 28 13:56:07 2017 From: jh at physics.ucf.edu (Joe Harrington) Date: Thu, 28 Sep 2017 13:56:07 -0400 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: (message from Warren Weckesser on Thu, 28 Sep 2017 11:21:50 -0400) Message-ID: All of this is orthogonal to the original and much more important point: We're not telling people HOW to fix their deprecated code. Perhaps an interested student could volunteer to make a page on scipy.org listing the deprecations and removals in each version (pulled from the release notes). Then, that person, developers, or other interested users could add modern alternatives and the rationale for removal. To the minor point about -Wd, is scipy turning off DeprecationWarnings by default, or is some environment like ipython doing it? I see numpy deprecation warnings all over the place, until I fix them. They're not turned off by default. I'm running in straight Python on Ubuntu Linux, with numpy and sometimes scipy. I'm not using ipython. This behavior has been true in all versions of Python, numpy, and scipy I've used. So, I think that paragraph could use another look, from the perspective of not just the preferred environment of the developer who wrote it. If SciPy turns off DeprecationWarnings and numpy doesn't, does this really make sense? Because numpy doesn't. If it's an environment doing it, then it should be noted that -Wd is needed only in those environments. Finally, it should be noted that -Wd is shorthand for -W default, which is all the Python man page talks about. Searching for -Wd there comes up with nothing. --jh-- On Thu, 28 Sep 2017 11:21:50 -0400, Warren Weckesser wrote: > On Thu, Sep 28, 2017 at 9:44 AM, Joe Harrington wrote: > > > Ok, I saw the -Wd at the end, but it just requests the default, so how > > does this change anything? > > > > Here's a fun sentence from the documentation at > https://docs.python.org/3.6/library/warnings.html#updating-code-for-new-versio> > ns-of-python > regarding the option -Wd: > > > This enables default handling for all warnings, including those that are > ignored by default. > > Fortunately, the sentences previous to that one explain that when -Wd is > used, DeprecationWarnings (usually ignored) are made visible. > > Warren > > > If it's just an ipython benefit, say that > > it's recommended for ipython users. > > > > Also, remember that for the vast majority of users, they don't install a > > new version, they suddenly discover that their system now has one, > > through some software update. Think students and casual users, as > > opposed to PhD research scientists experienced in software management. > > > > But, what we really need is more information about what to DO about the > > deprecations. Not, "we pulled the rug out from under you", but "so that > > we don't pull the rug out from under you, do this". And, in a > > more-obvious place than the release notes, which most users don't ever > > see. > > > > --jh-- > > > > >Um... This commit has to do with wheels, not deprecations. Great that > > >we're flying with wheels up, though. :-) > > > > > >--jh-- > > > > > >On Thu, 28 Sep 2017 21:48:41 +1300, Ralf Gommers > > > > >wrote: > > > > > >> Done in https://github.com/scipy/scipy/pull/7947 > > > > > >>> On Mon, Sep 18, 2017 at 1:16 AM, Joe Harrington > > >>> wrote: > > >>> > > >>>> On Sun, 17 Sep 2017 22:48:35 +1200 Ralf Gommers < > > ralf.gommers at gmail.com> > > >>>> wrote: > > >>>> > > >>>> > I'm excited to be able to announce the availability of the first > > beta > > >>>> > release of Scipy 1.0. > > >>>> > > > >>>> [I won't quote the entire release notes!] > > >>>> > > >>>> Awesome news, looooong in the making! > > >>>> > > >>>> The number of deprecations and removals is understandably large. > > >>>> I think we need a centralized resource for navigating deprecations, > > >>>> removals, and changes. It could be a simple web page, or a more > > >>>> involved database. It should list the change, give or link to the > > >>>> rationale, and state the date and package version number. Most > > >>>> importantly, it should give the suggested replacement(s). > > >>>> > > >>>> Googling "scipy deprecation" found only this page, plus a bunch of > > >>>> release notes and discussions. The page was last updated on Jan 16, > > >>>> 2017, and is limited to C-API deprecations: > > >>>> > > >>>> https://docs.scipy.org/doc/numpy-dev/reference/c-api. > > deprecations.html > > >>>> > > >>>> Even once such a document/page/searchable database exists, many will > > be > > >>>> unaware of it, or not know where to find it, so referring to it at the > > >>>> top of any list of deprecations/removals would be good. > > >>>> > > >>>> Once it exists, deprecations could be entered before being > > implemented, > > >>>> and deprecation warnings could refer to an index number in this > > >>>> resource, where the user could go for more information on how to fix > > the > > >>>> issue. > > >>>> > > >>> > > >>> Thanks for your thoughts Joe! I'm not sure a database is the way to go, > > >>> but you make a good point. We have to guide users a bit more in what > > to do > > >>> about deprecations. I think clear instructions about upgrading in both > > the > > >>> html docs and release announcements would be useful. I don't think > > there's > > >>> many users who run ``python -Wd my_code.py`` .(making depreacations > > that > > >>> affect your code visible) .... > > >>> > > >> > > > > > > > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > https://mail.python.org/mailman/listinfo/scipy-user From robert.kern at gmail.com Thu Sep 28 14:11:08 2017 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 28 Sep 2017 11:11:08 -0700 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On Thu, Sep 28, 2017 at 10:56 AM, Joe Harrington wrote: > To the minor point about -Wd, is scipy turning off DeprecationWarnings > by default, or is some environment like ipython doing it? It's a plain-old-python thing. ? python Enthought Deployment Manager -- https://www.enthought.com Python 2.7.13 |Enthought, Inc. (x86_64)| (default, Mar 2 2017, 08:20:50) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import warnings >>> warnings.filters [('ignore', None, , None, 0), ('ignore', None, , None, 0), ('ignore', None, , None, 0), ('ignore', None, , None, 0)] > I see numpy deprecation warnings all over the place, until I fix them. You might be thinking of the FutureWarnings, which are not ignored by default. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From ralf.gommers at gmail.com Thu Sep 28 21:08:59 2017 From: ralf.gommers at gmail.com (Ralf Gommers) Date: Fri, 29 Sep 2017 14:08:59 +1300 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: On Fri, Sep 29, 2017 at 6:56 AM, Joe Harrington wrote: > All of this is orthogonal to the original and much more important point: > We're not telling people HOW to fix their deprecated code. Eh, we do. The "deprecated features" section of https://github.com/scipy/scipy/blob/master/doc/release/1.0.0-notes.rst has details for most items. In case where that becomes impractical like for all the misc.pilutil functions, the deprecation warnings themselves will say what to do. Example: https://github.com/scipy/scipy/blob/master/scipy/misc/pilutil.py#L108 The only exceptions are for things we remove because they're just wrong, like bytescale: https://github.com/scipy/scipy/blob/master/scipy/misc/pilutil.py#L32. There we can't tell the user what to use, because it depends on the application - but almost certainly bytescale is not the right solution. Hence it's just removed. > Perhaps an > interested student could volunteer to make a page on scipy.org listing > the deprecations and removals in each version (pulled from the release > notes). Then, that person, developers, or other interested users could > add modern alternatives and the rationale for removal. > I'm -1 on a website for this on scipy.org, even if there was a volunteer. It's much less discoverable than having good warnings in the code, and will inevitably go out of date. Ralf > > To the minor point about -Wd, is scipy turning off DeprecationWarnings > by default, or is some environment like ipython doing it? > > I see numpy deprecation warnings all over the place, until I fix them. > They're not turned off by default. I'm running in straight Python on > Ubuntu Linux, with numpy and sometimes scipy. I'm not using ipython. > This behavior has been true in all versions of Python, numpy, and scipy > I've used. > > So, I think that paragraph could use another look, from the perspective > of not just the preferred environment of the developer who wrote it. If > SciPy turns off DeprecationWarnings and numpy doesn't, does this really > make sense? Because numpy doesn't. > > If it's an environment doing it, then it should be noted that -Wd is > needed only in those environments. Finally, it should be noted that -Wd > is shorthand for -W default, which is all the Python man page talks > about. Searching for -Wd there comes up with nothing. > > --jh-- > > On Thu, 28 Sep 2017 11:21:50 -0400, Warren Weckesser > wrote: > > > On Thu, Sep 28, 2017 at 9:44 AM, Joe Harrington > wrote: > > > > > Ok, I saw the -Wd at the end, but it just requests the default, so how > > > does this change anything? > > > > > > > > Here's a fun sentence from the documentation at > > https://docs.python.org/3.6/library/warnings.html# > updating-code-for-new-versio> > > ns-of-python > > regarding the option -Wd: > > > > > This enables default handling for all warnings, including those that > are > > ignored by default. > > > > Fortunately, the sentences previous to that one explain that when -Wd is > > used, DeprecationWarnings (usually ignored) are made visible. > > > > Warren > > > > > > If it's just an ipython benefit, say that > > > it's recommended for ipython users. > > > > > > Also, remember that for the vast majority of users, they don't install > a > > > new version, they suddenly discover that their system now has one, > > > through some software update. Think students and casual users, as > > > opposed to PhD research scientists experienced in software management. > > > > > > But, what we really need is more information about what to DO about the > > > deprecations. Not, "we pulled the rug out from under you", but "so > that > > > we don't pull the rug out from under you, do this". And, in a > > > more-obvious place than the release notes, which most users don't ever > > > see. > > > > > > --jh-- > > > > > > >Um... This commit has to do with wheels, not deprecations. Great > that > > > >we're flying with wheels up, though. :-) > > > > > > > >--jh-- > > > > > > > >On Thu, 28 Sep 2017 21:48:41 +1300, Ralf Gommers < > ralf.gommers at gmail.com > > > > > > > >wrote: > > > > > > > >> Done in https://github.com/scipy/scipy/pull/7947 > > > > > > > >>> On Mon, Sep 18, 2017 at 1:16 AM, Joe Harrington < > jh at physics.ucf.edu> > > > >>> wrote: > > > >>> > > > >>>> On Sun, 17 Sep 2017 22:48:35 +1200 Ralf Gommers < > > > ralf.gommers at gmail.com> > > > >>>> wrote: > > > >>>> > > > >>>> > I'm excited to be able to announce the availability of the first > > > beta > > > >>>> > release of Scipy 1.0. > > > >>>> > > > > >>>> [I won't quote the entire release notes!] > > > >>>> > > > >>>> Awesome news, looooong in the making! > > > >>>> > > > >>>> The number of deprecations and removals is understandably large. > > > >>>> I think we need a centralized resource for navigating > deprecations, > > > >>>> removals, and changes. It could be a simple web page, or a more > > > >>>> involved database. It should list the change, give or link to the > > > >>>> rationale, and state the date and package version number. Most > > > >>>> importantly, it should give the suggested replacement(s). > > > >>>> > > > >>>> Googling "scipy deprecation" found only this page, plus a bunch of > > > >>>> release notes and discussions. The page was last updated on Jan > 16, > > > >>>> 2017, and is limited to C-API deprecations: > > > >>>> > > > >>>> https://docs.scipy.org/doc/numpy-dev/reference/c-api. > > > deprecations.html > > > >>>> > > > >>>> Even once such a document/page/searchable database exists, many > will > > > be > > > >>>> unaware of it, or not know where to find it, so referring to it > at the > > > >>>> top of any list of deprecations/removals would be good. > > > >>>> > > > >>>> Once it exists, deprecations could be entered before being > > > implemented, > > > >>>> and deprecation warnings could refer to an index number in this > > > >>>> resource, where the user could go for more information on how to > fix > > > the > > > >>>> issue. > > > >>>> > > > >>> > > > >>> Thanks for your thoughts Joe! I'm not sure a database is the way > to go, > > > >>> but you make a good point. We have to guide users a bit more in > what > > > to do > > > >>> about deprecations. I think clear instructions about upgrading in > both > > > the > > > >>> html docs and release announcements would be useful. I don't think > > > there's > > > >>> many users who run ``python -Wd my_code.py`` .(making depreacations > > > that > > > >>> affect your code visible) .... > > > >>> > > > >> > > > > > > > > > > > _______________________________________________ > > > SciPy-User mailing list > > > SciPy-User at python.org > > > https://mail.python.org/mailman/listinfo/scipy-user > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjol at tjol.eu Fri Sep 29 05:02:52 2017 From: tjol at tjol.eu (Thomas Jollans) Date: Fri, 29 Sep 2017 11:02:52 +0200 Subject: [SciPy-User] ANN: SciPy 1.0 beta release In-Reply-To: References: Message-ID: <1acaf176-d3ba-df22-dacc-47cfae13b8e8@tjol.eu> On 2017-09-28 19:56, Joe Harrington wrote: > To the minor point about -Wd, is scipy turning off DeprecationWarnings > by default, or is some environment like ipython doing it? Python turns them off by default; ipython turns them ON (so a lot of users actually see them) > I see numpy deprecation warnings all over the place, until I fix them. > They're not turned off by default. I'm running in straight Python on > Ubuntu Linux, with numpy and sometimes scipy. I'm not using ipython. > This behavior has been true in all versions of Python, numpy, and scipy > I've used. That's interesting, since the default Pythons on Ubuntu don't show deprecation warnings: 0:tjol at krikkit:~% python Python 2.7.13 (default, Jan 19 2017, 14:48:08) [GCC 6.3.0 20170118] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import warnings >>> warnings.warn(DeprecationWarning('foo')) >>> 0:tjol at krikkit:~% python3 Python 3.5.3 (default, Jan 19 2017, 14:11:04) [GCC 6.3.0 20170118] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import warnings >>> warnings.warn(DeprecationWarning('foo')) >>> 0:tjol at krikkit:~% ipython Python 3.5.3 (default, Jan 19 2017, 14:11:04) Type 'copyright', 'credits' or 'license' for more information IPython 6.1.0 -- An enhanced Interactive Python. Type '?' for help. In [1]: import warnings In [2]: warnings.warn(DeprecationWarning('foo')) /home/tjol/.local/bin/ipython:1: DeprecationWarning: foo #!/usr/bin/python3 In [3]: Do you really want to exit ([y]/n)? 0:tjol at krikkit:~% python3 -Wd Python 3.5.3 (default, Jan 19 2017, 14:11:04) [GCC 6.3.0 20170118] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import warnings >>> warnings.warn(DeprecationWarning('foo')) __main__:1: DeprecationWarning: foo >>> > > So, I think that paragraph could use another look, from the perspective > of not just the preferred environment of the developer who wrote it. If > SciPy turns off DeprecationWarnings and numpy doesn't, does this really > make sense? Because numpy doesn't. > > If it's an environment doing it, then it should be noted that -Wd is > needed only in those environments. Finally, it should be noted that -Wd > is shorthand for -W default, which is all the Python man page talks > about. Searching for -Wd there comes up with nothing. > > --jh-- > > On Thu, 28 Sep 2017 11:21:50 -0400, Warren Weckesser > wrote: > >> On Thu, Sep 28, 2017 at 9:44 AM, Joe Harrington wrote: >> >>> Ok, I saw the -Wd at the end, but it just requests the default, so how >>> does this change anything? >> >> >> >> Here's a fun sentence from the documentation at >> https://docs.python.org/3.6/library/warnings.html#updating-code-for-new-versio> >> ns-of-python >> regarding the option -Wd: >> >>> This enables default handling for all warnings, including those that are >> ignored by default. >> >> Fortunately, the sentences previous to that one explain that when -Wd is >> used, DeprecationWarnings (usually ignored) are made visible. >> >> Warren >> >> >> If it's just an ipython benefit, say that >>> it's recommended for ipython users. >>> >>> Also, remember that for the vast majority of users, they don't install a >>> new version, they suddenly discover that their system now has one, >>> through some software update. Think students and casual users, as >>> opposed to PhD research scientists experienced in software management. >>> >>> But, what we really need is more information about what to DO about the >>> deprecations. Not, "we pulled the rug out from under you", but "so that >>> we don't pull the rug out from under you, do this". And, in a >>> more-obvious place than the release notes, which most users don't ever >>> see. >>> >>> --jh-- >>> >>>> Um... This commit has to do with wheels, not deprecations. Great that >>>> we're flying with wheels up, though. :-) >>>> >>>> --jh-- >>>> >>>> On Thu, 28 Sep 2017 21:48:41 +1300, Ralf Gommers >>> >>>> wrote: >>>> >>>>> Done in https://github.com/scipy/scipy/pull/7947 >>>> >>>>>> On Mon, Sep 18, 2017 at 1:16 AM, Joe Harrington >>>>>> wrote: >>>>>> >>>>>>> On Sun, 17 Sep 2017 22:48:35 +1200 Ralf Gommers < >>> ralf.gommers at gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> I'm excited to be able to announce the availability of the first >>> beta >>>>>>>> release of Scipy 1.0. >>>>>>>> >>>>>>> [I won't quote the entire release notes!] >>>>>>> >>>>>>> Awesome news, looooong in the making! >>>>>>> >>>>>>> The number of deprecations and removals is understandably large. >>>>>>> I think we need a centralized resource for navigating deprecations, >>>>>>> removals, and changes. It could be a simple web page, or a more >>>>>>> involved database. It should list the change, give or link to the >>>>>>> rationale, and state the date and package version number. Most >>>>>>> importantly, it should give the suggested replacement(s). >>>>>>> >>>>>>> Googling "scipy deprecation" found only this page, plus a bunch of >>>>>>> release notes and discussions. The page was last updated on Jan 16, >>>>>>> 2017, and is limited to C-API deprecations: >>>>>>> >>>>>>> https://docs.scipy.org/doc/numpy-dev/reference/c-api. >>> deprecations.html >>>>>>> >>>>>>> Even once such a document/page/searchable database exists, many will >>> be >>>>>>> unaware of it, or not know where to find it, so referring to it at the >>>>>>> top of any list of deprecations/removals would be good. >>>>>>> >>>>>>> Once it exists, deprecations could be entered before being >>> implemented, >>>>>>> and deprecation warnings could refer to an index number in this >>>>>>> resource, where the user could go for more information on how to fix >>> the >>>>>>> issue. >>>>>>> >>>>>> >>>>>> Thanks for your thoughts Joe! I'm not sure a database is the way to go, >>>>>> but you make a good point. We have to guide users a bit more in what >>> to do >>>>>> about deprecations. I think clear instructions about upgrading in both >>> the >>>>>> html docs and release announcements would be useful. I don't think >>> there's >>>>>> many users who run ``python -Wd my_code.py`` .(making depreacations >>> that >>>>>> affect your code visible) .... >>>>>> >>>>> >>>> >>>> >>> _______________________________________________ >>> SciPy-User mailing list >>> SciPy-User at python.org >>> https://mail.python.org/mailman/listinfo/scipy-user > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > From charlesr.harris at gmail.com Fri Sep 29 19:52:17 2017 From: charlesr.harris at gmail.com (Charles R Harris) Date: Fri, 29 Sep 2017 17:52:17 -0600 Subject: [SciPy-User] NumPy 1.13.3 released. Message-ID: HI All, On behalf of the NumPy team, I am pleased to annouce the release of Numpy 1.13.3. This is a re-release of 1.13.2, which suffered from compatibility problems, see issue 9786 . It is a bugfix release for some problems found since 1.13.1. The most important fixes are for CVE-2017-12852 and the new temporary elision. Users of earlier versions of 1.13 should upgrade. The Python versions supported are 2.7 and 3.4 - 3.6. The Python 3.6 wheels available from PIP are built with Python 3.6.2 and should be compatible with all previous versions of Python 3.6. It was cythonized with Cython 0.26.1, which should be free of the bugs found in 0.27 while also being compatible with Python 3.7-dev. The Windows wheels were built with OpenBlas instead ATLAS, which should improve the performance of the linear algebra functions. Wheels and zip archives are available from PyPI , both zip and tar archives are available from Github . Contributors ============ A total of 12 people contributed to this release. People with a "+" by their names contributed a patch for the first time. * Allan Haldane * Brandon Carter * Charles Harris * Eric Wieser * Iryna Shcherbina + * James Bourbeau + * Jonathan Helmus * Julian Taylor * Matti Picus * Michael Lamparski + * Michael Seifert * Ralf Gommers Pull requests merged ==================== A total of 20 pull requests were merged for this release. * #9390 BUG: Return the poly1d coefficients array directly * #9555 BUG: Fix regression in 1.13.x in distutils.mingw32ccompiler. * #9556 BUG: Fix true_divide when dtype=np.float64 specified. * #9557 DOC: Fix some rst markup in numpy/doc/basics.py. * #9558 BLD: Remove -xhost flag from IntelFCompiler. * #9559 DOC: Removes broken docstring example (source code, png, pdf)... * #9580 BUG: Add hypot and cabs functions to WIN32 blacklist. * #9732 BUG: Make scalar function elision check if temp is writeable. * #9736 BUG: Various fixes to np.gradient * #9742 BUG: Fix np.pad for CVE-2017-12852 * #9744 BUG: Check for exception in sort functions, add tests * #9745 DOC: Add whitespace after "versionadded::" directive so it actually... * #9746 BUG: Memory leak in np.dot of size 0 * #9747 BUG: Adjust gfortran version search regex * #9757 BUG: Cython 0.27 breaks NumPy on Python 3. * #9764 BUG: Ensure `_npy_scaled_cexp{,f,l}` is defined when needed. * #9765 BUG: PyArray_CountNonzero does not check for exceptions * #9766 BUG: Fixes histogram monotonicity check for unsigned bin values * #9767 BUG: Ensure consistent result dtype of count_nonzero * #9771 BUG, MAINT: Fix mtrand for Cython 0.27. Enjoy Chuck -------------- next part -------------- An HTML attachment was scrubbed... URL: