Mailman 3 January 2015 - SciPy-Dev

Adding routines for updating QR decompositions
by Eric Moore 06 Apr '15

06 Apr '15

Hi, I've just submitted PR #4249 adding qr_update, qr_insert and qr_delete for performing rank-k updates, adding rows or columns and removing rows or colums. https://github.com/scipy/scipy/pull/4249 Comments welcome. Eric.

5 6

Least-Squares Linear Solver ( scipy.linalg.lstsq ) not optimal
by Alexander Grigorievskiy 09 Feb '15

09 Feb '15

Hello Scipy, I have been elaborating least-squares solvers recently I found out that the current implementation which is inside scipy.linalg.lstsq is not optimal. Of course, I may be wrong but here are my arguments: 1) Intrinsically it calls the LAPACK function "gelss" which uses SVD (as written in the LAPACK documentation). There are, however, two more routines which solve the same problem "gelsy" (uses complete orthogonal factorization) and "gelsd" (uses divide-and-conquer SVD algorithm). In the LAPACK documentation it is written that "gelsd" work faster then "gelss". http://www.netlib.org/lapack/lug/node27.html 2) In Numpy there is the same function "lstsq" and they call "gelsd" there. 3) I run my own tests analyzing the speed of four functions: "gelss" (the current one used in scipy.linalg.lstsq) "gelsy" (present in LAPACK uses complete orthogonal factorization) "gelsd" (present in LAPACK uses divide-and-conquer SVD) numpy.lstsq ( uses "gelsd" from lapack_lite) I have imported the missing functions (from LAPACK) in Scipy by including them into the file scipy/linalg/flapack.pyf.src, recompiling scipy, and creating almost the same function as "lstsq" but calling different LAPACK functions. The file with the test results is in the attachment. Based on this the fastest method is "gelsy" shortly followed by "gelsd". I want still to run couple more tests regarding how fast the methods scale if you change only one dimension of the matrix, or you can propose some other tests. I also monitored accuracies but they are almost the same, I can send the plot if anyone is interested. I would like to propose to change the current implementation to call "gelsy" or "gelsd" from LAPACK along with corresponding modifications. Best Regards, Alexander Grigorevskiy, PhD student, Aalto University.

7 37

Regarding involvement in Scipy projects
by dp docs 27 Jan '15

27 Jan '15

Dear Developers, I believe that I have good knowledge of C/C++, python programming languages. I have read the project Ideas from the idea page and most of them seems me interesting. But I have no clue from where should I start. Can anyone please provide me a sequence wise guideline so that I can be able to get involved successfully in the development in this open source projects. Please provide the link of the Source code also which I need to modify or re factor in case if it is better to start from re factoring. i shall be very much grateful to you for your effort. Thanks Durgesh Pandey.

3 3

Update NumPy for Matlab Users
by Joseph Martinot-Lagarde 26 Jan '15

26 Jan '15

Hi, I noticed some problems in the Wiki page "NumPy for Matlab Users" [1], but I dan't edit it myself. Here is what i found: - For symbolic calculation, swiginac is the proposed solution, but it has not been updated on pypi since 2007, and the home page is down. I guess that the standard is now sympy. - There are multiple dead links on the page. It can be easily see on the general informations about this page [2]. - On the pros and cons to use array or matrix in numpy about dot(), the form dot(dot(A,B),C) should be replaced by the more readeable A.dot(B).dot(C). Also this will have to change when the '@' operator is introduced in python 3.5. - In "Customizing Your Environment", numpy is imported as "num", while the commonly used name is "np". THis page is really a great help for Matlab users, keeping it up to date is really important ! Joseph [1] http://wiki.scipy.org/NumPy_for_Matlab_Users [2] http://wiki.scipy.org/NumPy_for_Matlab_Users?action=info&general=1

1 0

scipy minimize with bound
by Martin Hofsäß 26 Jan '15

26 Jan '15

Hi all, I want to uses the minimize function with bounds, but I get an error: ValueError: _lbfgsb.setulb() 13rd argument (iprint) can't be converted to int I call the function like this: spo.minimize(residuals,(3.9,40.,0.2),args=(k,puu,pvv,pww,cuw,stat),method='L-BFGS-B',options={'disp':'True','maxiter':100},tol=5.0e-7,bounds=((0,None),(0,None),(0,None))) TNC and SLSQP worked with bounds. I'm using linux 64bit python 2.7.3 with scipy 0.14.0. Thanks for your help.

2 2

CSD in scipy.signal?
by Eric Quintero 21 Jan '15

21 Jan '15

Hi all, I would like to write cross-spectral density (and, by extension, coherence) methods for scipy.signal, as a complement the welch’s PSD method already there. I haven’t contributed to scipy before, and the website encourages discussion of new features on this list, so here I am. In addition, when computing PSDs in my field, we sometimes prefer to take the median of the segments, rather than the mean, in order to reduce the effect of transients in the data when trying to evaluate a stationary noise floor. I would like to add this option as a kwarg. Thanks for your time, Eric Q.

2 1

ANN: Scipy 0.15.1
by Pauli Virtanen 18 Jan '15

18 Jan '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dear all, We are pleased to announce the Scipy 0.15.1 release. Scipy 0.15.1 contains only bugfixes. The module ``scipy.linalg.calc_lwork`` removed in Scipy 0.15.0 is restored. This module is not a part of Scipy's public API, and although it is available again in Scipy 0.15.1, using it is deprecated and it may be removed again in a future Scipy release. Source tarballs, binaries, and full release notes are available at https://sourceforge.net/projects/scipy/files/scipy/0.15.1/ Best regards, Pauli Virtanen ========================== SciPy 0.15.1 Release Notes ========================== SciPy 0.15.1 is a bug-fix release with no new features compared to 0.15.0. Issues fixed - ------------ * `#4413 <https://github.com/scipy/scipy/pull/4413>`__: BUG: Tests too strict, f2py doesn't have to overwrite this array * `#4417 <https://github.com/scipy/scipy/pull/4417>`__: BLD: avoid using NPY_API_VERSION to check not using deprecated... * `#4418 <https://github.com/scipy/scipy/pull/4418>`__: Restore and deprecate scipy.linalg.calc_work -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlS8CA4ACgkQ6BQxb7O0pWCmOQCgzg9AXDaqRaK5/QBWopIrv2OA WkEAn0ltDfDHFpw0zMzB9mUscAAb2xnE =JrGj -----END PGP SIGNATURE-----

1 0

error in docstring of pdist
by Pierre Haessig 13 Jan '15

13 Jan '15

Hello, It's my first use of the distance function from scipy.spatial, and I wonder if there is an error in the docstring of pdist: http://docs.scipy.org/doc/scipy-0.14.0/reference/generated/scipy.spatial.di… (same thing for cdist) [...] Returns ------- Y : ndarray Returns a condensed distance matrix Y. For each :math:`i` and :math:`j` (where :math:`i<j<n`), the metric ``dist(u=X[i], v=X[j])`` is computed and stored in entry ``ij``. [...] Is this indeed the entry i*j ? Because the docstring of squareform says : [...] The X[i, j] and X[j, i] values are set to v[{n \choose 2}-{n-i \choose 2} + (j-u-1)] [...] best, Pierre

2 2

ANN: Scipy 0.15.0 release
by Pauli Virtanen 11 Jan '15

11 Jan '15

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dear all, We are pleased to announce the Scipy 0.15.0 release. The 0.15.0 release contains bugfixes and new features, most important of which are mentioned in the excerpt from the release notes below. Source tarballs, binaries, and full release notes are available at https://sourceforge.net/projects/scipy/files/scipy/0.15.0/ Best regards, Pauli Virtanen ========================== SciPy 0.15.0 Release Notes ========================== SciPy 0.15.0 is the culmination of 6 months of hard work. It contains several new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Moreover, our development attention will now shift to bug-fix releases on the 0.16.x branch, and on adding new features on the master branch. This release requires Python 2.6, 2.7 or 3.2-3.4 and NumPy 1.5.1 or greater. New features ============ Linear Programming Interface - ---------------------------- The new function `scipy.optimize.linprog` provides a generic linear programming similar to the way `scipy.optimize.minimize` provides a generic interface to nonlinear programming optimizers. Currently the only method supported is *simplex* which provides a two-phase, dense-matrix-based simplex algorithm. Callbacks functions are supported, allowing the user to monitor the progress of the algorithm. Differential evolution, a global optimizer - ------------------------------------------ A new `scipy.optimize.differential_evolution` function has been added to the ``optimize`` module. Differential Evolution is an algorithm used for finding the global minimum of multivariate functions. It is stochastic in nature (does not use gradient methods), and can search large areas of candidate space, but often requires larger numbers of function evaluations than conventional gradient based techniques. ``scipy.signal`` improvements - ----------------------------- The function `scipy.signal.max_len_seq` was added, which computes a Maximum Length Sequence (MLS) signal. ``scipy.integrate`` improvements - -------------------------------- It is now possible to use `scipy.integrate` routines to integrate multivariate ctypes functions, thus avoiding callbacks to Python and providing better performance. ``scipy.linalg`` improvements - ----------------------------- The function `scipy.linalg.orthogonal_procrustes` for solving the procrustes linear algebra problem was added. BLAS level 2 functions ``her``, ``syr``, ``her2`` and ``syr2`` are now wrapped in ``scipy.linalg``. ``scipy.sparse`` improvements - ----------------------------- `scipy.sparse.linalg.svds` can now take a ``LinearOperator`` as its main input. ``scipy.special`` improvements - ------------------------------ Values of ellipsoidal harmonic (i.e. Lame) functions and associated normalization constants can be now computed using ``ellip_harm``, ``ellip_harm_2``, and ``ellip_normal``. New convenience functions ``entr``, ``rel_entr`` ``kl_div``, ``huber``, and ``pseudo_huber`` were added. ``scipy.sparse.csgraph`` improvements - ------------------------------------- Routines ``reverse_cuthill_mckee`` and ``maximum_bipartite_matching`` for computing reorderings of sparse graphs were added. ``scipy.stats`` improvements - ---------------------------- Added a Dirichlet multivariate distribution, `scipy.stats.dirichlet`. The new function `scipy.stats.median_test` computes Mood's median test. The new function `scipy.stats.combine_pvalues` implements Fisher's and Stouffer's methods for combining p-values. `scipy.stats.describe` returns a namedtuple rather than a tuple, allowing users to access results by index or by name. Deprecated features =================== The `scipy.weave` module is deprecated. It was the only module never ported to Python 3.x, and is not recommended to be used for new code - use Cython instead. In order to support existing code, ``scipy.weave`` has been packaged separately: https://github.com/scipy/weave. It is a pure Python package, and can easily be installed with ``pip install weave``. `scipy.special.bessel_diff_formula` is deprecated. It is a private function, and therefore will be removed from the public API in a following release. ``scipy.stats.nanmean``, ``nanmedian`` and ``nanstd`` functions are deprecated in favor of their numpy equivalents. Backwards incompatible changes ============================== scipy.ndimage - ------------- The functions `scipy.ndimage.minimum_positions`, `scipy.ndimage.maximum_positions`` and `scipy.ndimage.extrema` return positions as ints instead of floats. scipy.integrate - --------------- The format of banded Jacobians in `scipy.integrate.ode` solvers is changed. Note that the previous documentation of this feature was erroneous. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlSyt/cACgkQ6BQxb7O0pWA8SACfXmpUsJcXT5espj71OYpeaj5b JJwAoL10ud3q1f51A5Ij4lgqMeZGnHlj =ZmOl -----END PGP SIGNATURE-----

1 0

Re: [SciPy-Dev] SciPy-Dev Digest, Vol 135, Issue 2
by Matt Newville 08 Jan '15

08 Jan '15

Hi Andrew, leastsq() can be halted by having your objective function return None. I think that doesn't work for the scalar minimizers, though I didn't try them all. Returning None from the objective function is not very fine-grained (you wouldn't know why it happened), but it seems easier to have the wrappers around the Fortran code look for a NULL return value than for a HaltError exception. It seems like a sensible, if crude, way to say "Stop now", and might be worth considering for all the minimizing functions. --Matt

1 0