Mailman 3 September 2017 - NumPy-Discussion

floor with dtype
by Renato Fabbri 25 Sep '17

25 Sep '17

""" In [3]: n.floor(n.linspace(0,5,7), dtype=n.int) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-3-1471f5d738e8> in <module>() ----> 1 n.floor(n.linspace(0,5,7), dtype=n.int) TypeError: No loop matching the specified signature and casting was found for ufunc floor In [4]: n.__version__ Out[4]: '1.11.0' """ Is this the expected behavior? I am doing: >>> myints = n.array(n.floor(myarray), dtype=n.int) to get the integers. tx. R. -- Renato Fabbri GNU/Linux User #479299 labmacambira.sourceforge.net

2 2

I love integers
by josef.pktd＠gmail.com 21 Sep '17

21 Sep '17

After many hours of debugging deep inside a hessian that sometimes produced weird results. In [48]: (-y**2 * (y - 1)) / (y**2. * (y - 1)) Out[48]: array([[ -1.00000000e+00], [ -1.77041643e-03], [ 5.80863636e-04], [ -5.37729923e-03], [ -1.74809893e-03], [ 2.25499819e-02], [ 8.65999453e-03]]) y is int32 Josef https://github.com/statsmodels/statsmodels/issues/3919

1 0

converting list of int16 values to bitmask and back to list of int32\float values
by Nissim Derdiger 20 Sep '17

20 Sep '17

Hi all! I'm writing a Modbus TCP client using pymodbus3 library. When asking for some parameters, the response is always a list of int16. In order to make the values usable, I need to transfer them into 32bit bites, than put them in the correct order (big\little endian wise), and then to cast them back to the desired format (usually int32 or float) I've solved it with a pretty naïve code, but I'm guessing there must be a more elegant and fast way to solve it with NumPy. Your help would be very much appreciated! Nissim. My code: def Read(StartAddress, NumOfRegisters, FunctionCode,ParameterType,BitOrder): # select the Parameters format PrmFormat = 'f' # default is float if ParameterType == 'int': PrmFormat = 'i' # select the endian state - maybe move to the connect function? endian = '<I' # default is big endian if BitOrder == 'little': endian = '>I' # start asking for the payload payload = None while payload == None: payload = Modbus.read_input_registers(StartAddress, NumOfRegisters) #### parse the answer ResultRegisters = [] # convert the returned registers from list of int16 to list of 32 bits bitmaks for reg in range(int(NumOfRegisters / 2)): ResultRegisters[reg] = struct.pack(endian, payload.registers[2 * reg]) + struct.pack(endian,payload.registers[2 * reg + 1]) # convert this list to a list with the real parameter format for reg in range(len(ResultRegisters)): ResultRegisters[reg]= struct.unpack(PrmFormat,ResultRegisters(reg)) # return results return ResultRegisters

2 1

ANN: SfePy 2017.3
by Robert Cimrman 19 Sep '17

19 Sep '17

I am pleased to announce release 2017.3 of SfePy. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (limited support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: https://mail.python.org/mm3/mailman3/lists/sfepy.python.org/ Git (source) repository, issue tracker: https://github.com/sfepy/sfepy Highlights of this release -------------------------- - support preconditioning in SciPy and PyAMG based linear solvers - user-defined preconditioners for PETSc linear solvers - parallel multiscale (macro-micro) homogenization-based computations - improved tutorial and installation instructions For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Cheers, Robert Cimrman --- Contributors to this release in alphabetical order: Robert Cimrman Lubos Kejzlar Vladimir Lukes Matyas Novak

1 0

Re: [Numpy-discussion] [SciPy-User] ANN: SciPy 1.0 beta release
by Ralf Gommers 19 Sep '17

19 Sep '17

On Mon, Sep 18, 2017 at 10:36 PM, Matthew Brett <matthew.brett(a)gmail.com> wrote: > Hi, > > On Mon, Sep 18, 2017 at 11:14 AM, Ralf Gommers <ralf.gommers(a)gmail.com> > wrote: > > > > > > On Mon, Sep 18, 2017 at 10:11 PM, Matthew Brett <matthew.brett(a)gmail.com > > > > wrote: > >> > >> Hi, > >> > >> On Mon, Sep 18, 2017 at 11:07 AM, Thomas Kluyver <takowl(a)gmail.com> > wrote: > >> > On 18 September 2017 at 10:59, Ralf Gommers <ralf.gommers(a)gmail.com> > >> > wrote: > >> >> > >> >> Binary wheels for Windows, Linux and OS X (for all supported Python > >> >> versions, 32-bit and 64-bit) can be found at http://wheels.scipy.org > . > >> >> To > >> >> install directly with pip: > >> >> > >> >> pip install scipy=='1.0.0b1' -f http://wheels.scipy.org > >> >> --trusted-host > >> >> wheels.scipy.org > >> > > >> > > >> > I don't want to criticise the hard work that has gone into making this > >> > available, but I'm disappointed that we're telling people to install > >> > software over an insecure HTTP connection. > >> > >> I personally prefer the following recipe: > >> > >> pip install -f > >> https://3f23b170c54c2533c070-1c8a9b3114517dc5fe17b7c3f8c63a4 > 3.ssl.cf2.rackcdn.com > >> scipy=='1.0.0b1' > >> > >> > Can the wheels not be uploaded to PyPI? > >> > >> Sounds like a good idea. I can do that - any objections? > > > > > > That would be helpful Matthew, I'm about to sign off for today. > > Done - new instructions for testing: > > pip install --pre --upgrade scipy > Thanks Matthew! Replying to all lists with the better install instructions. Cheers, Ralf

1 0

ANN: SciPy 1.0 beta release
by Ralf Gommers 18 Sep '17

18 Sep '17

Hi all, I'm excited to be able to announce the availability of the first beta release of Scipy 1.0. This is a big release, and a version number that has been 16 years in the making. It contains a few more deprecations and backwards incompatible changes than an average release. Therefore please do test it on your own code, and report any issues on the Github issue tracker or on the scipy-dev mailing list. Sources: https://github.com/scipy/scipy/releases/tag/v1.0.0b1 Binary wheels: will follow tomorrow, I'll announce those when ready (TravisCI is under maintenance right now) Thanks to everyone who contributed to this release! Ralf Release notes (full notes including authors, closed issued and merged PRs at the Github Releases link above): ========================== SciPy 1.0.0 Release Notes ========================== .. note:: Scipy 1.0.0 is not released yet! .. contents:: SciPy 1.0.0 is the culmination of 8 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Moreover, our development attention will now shift to bug-fix releases on the 1.0.x branch, and on adding new features on the master branch. Some of the highlights of this release are: - Major build improvements. Windows wheels are available on PyPI for the first time, and continuous integration has been set up on Windows and OS X in addition to Linux. - A set of new ODE solvers and a unified interface to them (`scipy.integrate.solve_ivp`). - Two new trust region optimizers and a new linear programming method, with improved performance compared to what `scipy.optimize` offered previously. - Many new BLAS and LAPACK functions were wrapped. The BLAS wrappers are now complete. This release requires Python 2.7 or 3.4+ and NumPy 1.8.2 or greater. This is also the last release to support LAPACK 3.1.x - 3.3.x. Moving the lowest supported LAPACK version to >3.2.x was long blocked by Apple Accelerate providing the LAPACK 3.2.1 API. We have decided that it's time to either drop Accelerate or, if there is enough interest, provide shims for functions added in more recent LAPACK versions so it can still be used. New features ============ `scipy.cluster` improvements ---------------------------- `scipy.cluster.hierarchy.optimal_leaf_ordering`, a function to reorder a linkage matrix to minimize distances between adjacent leaves, was added. `scipy.fftpack` improvements ---------------------------- N-dimensional versions of the discrete sine and cosine transforms and their inverses were added as ``dctn``, ``idctn``, ``dstn`` and ``idstn``. `scipy.integrate` improvements ------------------------------ A set of new ODE solvers have been added to `scipy.integrate`. The convenience function `scipy.integrate.solve_ivp` allows uniform access to all solvers. The individual solvers (``RK23``, ``RK45``, ``Radau``, ``BDF`` and ``LSODA``) can also be used directly. `scipy.linalg` improvements ---------------------------- The BLAS wrappers in `scipy.linalg.blas` have been completed. Added functions are ``*gbmv``, ``*hbmv``, ``*hpmv``, ``*hpr``, ``*hpr2``, ``*spmv``, ``*spr``, ``*tbmv``, ``*tbsv``, ``*tpmv``, ``*tpsv``, ``*trsm``, ``*trsv``, ``*sbmv``, ``*spr2``, Wrappers for the LAPACK functions ``*gels``, ``*stev``, ``*sytrd``, ``*hetrd``, ``*sytf2``, ``*hetrf``, ``*sytrf``, ``*sycon``, ``*hecon``, ``*gglse``, ``*stebz``, ``*stemr``, ``*sterf``, and ``*stein`` have been added. The function `scipy.linalg.subspace_angles` has been added to compute the subspace angles between two matrices. The function `scipy.linalg.clarkson_woodruff_transform` has been added. It finds low-rank matrix approximation via the Clarkson-Woodruff Transform. The functions `scipy.linalg.eigh_tridiagonal` and `scipy.linalg.eigvalsh_tridiagonal`, which find the eigenvalues and eigenvectors of tridiagonal hermitian/symmetric matrices, were added. `scipy.ndimage` improvements ---------------------------- Support for homogeneous coordinate transforms has been added to `scipy.ndimage.affine_transform`. The ``ndimage`` C code underwent a significant refactoring, and is now a lot easier to understand and maintain. `scipy.optimize` improvements ----------------------------- The methods ``trust-region-exact`` and ``trust-krylov`` have been added to the function `scipy.optimize.minimize`. These new trust-region methods solve the subproblem with higher accuracy at the cost of more Hessian factorizations (compared to dogleg) or more matrix vector products (compared to ncg) but usually require less nonlinear iterations and are able to deal with indefinite Hessians. They seem very competitive against the other Newton methods implemented in scipy. `scipy.optimize.linprog` gained an interior point method. Its performance is superior (both in accuracy and speed) to the older simplex method. `scipy.signal` improvements --------------------------- An argument ``fs`` (sampling frequency) was added to the following functions: ``firwin``, ``firwin2``, ``firls``, and ``remez``. This makes these functions consistent with many other functions in `scipy.signal` in which the sampling frequency can be specified. `scipy.signal.freqz` has been sped up significantly for FIR filters. `scipy.sparse` improvements --------------------------- Iterating over and slicing of CSC and CSR matrices is now faster by up to ~35%. The ``tocsr`` method of COO matrices is now several times faster. The ``diagonal`` method of sparse matrices now takes a parameter, indicating which diagonal to return. `scipy.sparse.linalg` improvements ---------------------------------- A new iterative solver for large-scale nonsymmetric sparse linear systems, `scipy.sparse.linalg.gcrotmk`, was added. It implements ``GCROT(m,k)``, a flexible variant of ``GCROT``. `scipy.sparse.linalg.lsmr` now accepts an initial guess, yielding potentially faster convergence. SuperLU was updated to version 5.2.1. `scipy.spatial` improvements ---------------------------- Many distance metrics in `scipy.spatial.distance` gained support for weights. The signatures of `scipy.spatial.distance.pdist` and `scipy.spatial.distance.cdist` were changed to ``*args, **kwargs`` in order to support a wider range of metrics (e.g. string-based metrics that need extra keywords). Also, an optional ``out`` parameter was added to ``pdist`` and ``cdist`` allowing the user to specify where the resulting distance matrix is to be stored `scipy.stats` improvements -------------------------- The methods ``cdf`` and ``logcdf`` were added to `scipy.stats.multivariate_normal`, providing the cumulative distribution function of the multivariate normal distribution. New statistical distance functions were added, namely `scipy.stats.wasserstein_distance` for the first Wasserstein distance and `scipy.stats.energy_distance` for the energy distance. Deprecated features =================== The following functions in `scipy.misc` are deprecated: ``bytescale``, ``fromimage``, ``imfilter``, ``imread``, ``imresize``, ``imrotate``, ``imsave``, ``imshow`` and ``toimage``. Most of those functions have unexpected behavior (like rescaling and type casting image data without the user asking for that). Other functions simply have better alternatives. ``scipy.interpolate.interpolate_wrapper`` and all functions in that submodule are deprecated. This was a never finished set of wrapper functions which is not relevant anymore. The ``fillvalue`` of `scipy.signal.convolve2d` will be cast directly to the dtypes of the input arrays in the future and checked that it is a scalar or an array with a single element. Backwards incompatible changes ============================== The following deprecated functions have been removed from `scipy.stats`: ``betai``, ``chisqprob``, ``f_value``, ``histogram``, ``histogram2``, ``pdf_fromgamma``, ``signaltonoise``, ``square_of_sums``, ``ss`` and ``threshold``. The following deprecated functions have been removed from `scipy.stats.mstats`: ``betai``, ``f_value_wilks_lambda``, ``signaltonoise`` and ``threshold``. The deprecated ``a`` and ``reta`` keywords have been removed from `scipy.stats.shapiro`. The deprecated functions ``sparse.csgraph.cs_graph_components`` and ``sparse.linalg.symeig`` have been removed from `scipy.sparse`. The following deprecated keywords have been removed in `scipy.sparse.linalg`: ``drop_tol`` from ``splu``, and ``xtype`` from ``bicg``, ``bicgstab``, ``cg``, ``cgs``, ``gmres``, ``qmr`` and ``minres``. The deprecated functions ``expm2`` and ``expm3`` have been removed from `scipy.linalg`. The deprecated keyword ``q`` was removed from `scipy.linalg.expm`. And the deprecated submodule ``linalg.calc_lwork`` was removed. The deprecated functions ``C2K``, ``K2C``, ``F2C``, ``C2F``, ``F2K`` and ``K2F`` have been removed from `scipy.constants`. The deprecated ``ppform`` class was removed from `scipy.interpolate`. The deprecated keyword ``iprint`` was removed from `scipy.optimize.fmin_cobyla`. The default value for the ``zero_phase`` keyword of `scipy.signal.decimate` has been changed to True. The ``kmeans`` and ``kmeans2`` functions in `scipy.cluster.vq` changed the method used for random initialization, so using a fixed random seed will not necessarily produce the same results as in previous versions. `scipy.special.gammaln` does not accept complex arguments anymore. The deprecated functions ``sph_jn``, ``sph_yn``, ``sph_jnyn``, ``sph_in``, ``sph_kn``, and ``sph_inkn`` have been removed. Users should instead use the functions ``spherical_jn``, ``spherical_yn``, ``spherical_in``, and ``spherical_kn``. Be aware that the new functions have different signatures. The cross-class properties of `scipy.signal.lti` systems have been removed. The following properties/setters have been removed: Name - (accessing/setting has been removed) - (setting has been removed) * StateSpace - (``num``, ``den``, ``gain``) - (``zeros``, ``poles``) * TransferFunction (``A``, ``B``, ``C``, ``D``, ``gain``) - (``zeros``, ``poles``) * ZerosPolesGain (``A``, ``B``, ``C``, ``D``, ``num``, ``den``) - () ``signal.freqz(b, a)`` with ``b`` or ``a`` >1-D raises a ``ValueError``. This was a corner case for which it was unclear that the behavior was well-defined. The method ``var`` of `scipy.stats.dirichlet` now returns a scalar rather than an ndarray when the length of alpha is 1. Other changes ============= SciPy now has a formal governance structure. It consists of a BDFL (Pauli Virtanen) and a Steering Committee. See `the governance document < https://github.com/scipy/scipy/blob/master/doc/source/dev/governance/govern… >`_ for details. It is now possible to build SciPy on Windows with MSVC + gfortran! Continuous integration has been set up for this build configuration on Appveyor, building against OpenBLAS. Continuous integration for OS X has been set up on TravisCI. The SciPy test suite has been migrated from ``nose`` to ``pytest``. ``scipy/_distributor_init.py`` was added to allow redistributors of SciPy to add custom code that needs to run when importing SciPy (e.g. checks for hardware, DLL search paths, etc.). Support for PEP 518 (specifying build system requirements) was added - see ``pyproject.toml`` in the root of the SciPy repository. In order to have consistent function names, the function ``scipy.linalg.solve_lyapunov`` is renamed to `scipy.linalg.solve_continuous_lyapunov`. The old name is kept for backwards-compatibility.

4 5

Dropping support for Accelerate
by Ilhan Polat 16 Sep '17

16 Sep '17

A few months ago, I had the innocent intention to wrap LDLt decomposition routines of LAPACK into SciPy but then I am made aware that the minimum required version of LAPACK/BLAS was due to Accelerate framework. Since then I've been following the core SciPy team and others' discussion on this issue. We have been exchanging opinions for quite a while now within various SciPy issues and PRs about the ever-increasing Accelerate-related issues and I've compiled a brief summary about the ongoing discussions to reduce the clutter. First, I would like to kindly invite everyone to contribute and sharpen the cases presented here https://github.com/scipy/scipy/wiki/Dropping-support-for-Accelerate The reason I specifically wanted to post this also in NumPy mailing list is to probe for the situation from the NumPy-Accelerate perspective. Is there any NumPy specific problem that would indirectly effect SciPy should the support for Accelerate is dropped?

5 16

Re: [Numpy-discussion] Only integer scalar arrays can be converted to a scalar index
by Michael Bostock 16 Sep '17

16 Sep '17

I was hoping that numpy doing this in a vectorised way would only load the surrounding traces into memory for each X and Y as it needs to rather than the whole cube. I'm using hdf5 for the storage. My example was just a short example without using hdf5. On 15 Sep 2017 1:16 am, "Elliot Hallmark" <Permafacture(a)gmail.com> wrote: Won't any solution not using hdf5 or some other chunked on disk storage method load the whole cube into memory? _______________________________________________ NumPy-Discussion mailing list NumPy-Discussion(a)python.org https://mail.python.org/mailman/listinfo/numpy-discussion

6 6

Questions on np.piecewise
by Joe 15 Sep '17

15 Sep '17

Hello, I have two questions and hope that you can help me. 1.) Is np.piecewise only defined for two conditions or should something like [0 < x <= 90, 90 < x <= 180, 180 < x <= 270] also work? 2.) Why does np.piecewise(np.array([50]), [0 < x <= 90, 90 < x <= 180], [1.1, 2.1]) return [2] and not [2.1] ? Kind regards, Joe

2 2

Only integer scalar arrays can be converted to a scalar index
by Michael Bostock 15 Sep '17

15 Sep '17

Hi, I am trying to do a slding window on a cube (3D array) to get the average over a block of vertical 1D arrays. I have achieved this using the stride_tricks.asstrided but this will load the whole cube into memory at once and is not suitable for large cubes. I have also achieved it using an nditer but this is a lot slower: def trace_block(vol, x, y, half_window): return vol[x - half_window:x + half_window + 1, y - half_window:y + half_window + 1] vol = np.linspace(1, 125, 125, dtype=np.int32).reshape(5, 5, 5) window_size = 3 x, y, z = vol.shape half_window = (window_size - 1) // 2 xs = np.arange(half_window, x - half_window, dtype=np.int16) ys = np.arange(half_window, y - half_window, dtype=np.int16) averaged = np.zeros((5, 5, 5)) for x, y in np.nditer(np.ix_(xs, ys)): averaged[x, y] = np.mean(trace_block(vol, x, y, half_window), (0, 1)) My attempt at using numpy vectorisation to avoid the for loop throws the error in the subject: vol = np.linspace(1, 125, 125, dtype=np.int32).reshape(5, 5, 5) window_size = 3 x, y, z = vol.shape half_window = (window_size - 1) // 2 xs = np.arange(half_window, x - half_window, dtype=np.int16) ys = np.arange(half_window, y - half_window, dtype=np.int16) averaged = np.zeros((5, 5, 5)) xi, yi = np.ix_(xs, ys) averaged[xi, yi] = np.mean(trace_block(vol, xi, yi, half_window), (0, 1)) Is there any way to do slicing as shown in the trace_block function to support the xi and yi grid arrays? Any help you can provide will be greatly appreciated.

2 1