Mailman 3 October 2013 - NumPy-Discussion

Code review request: PrintOptions
by Neil Girdhar 28 Oct '13

28 Oct '13

This is my first code review request, so I may have done some things wrong. I think the following URL should work? https://github.com/MisterSheik/numpy/compare Best, Neil

3 4

Testing
by Neil Girdhar 28 Oct '13

28 Oct '13

How do I test a patch that I've made locally? I can't seem to import numpy locally: Error importing numpy: you should not try to import numpy from its source directory; please exit the numpy source tree, and relaunch your python intepreter from there.

3 4

A context manager for print options
by Neil Girdhar 27 Oct '13

27 Oct '13

Why not replace get_printoptions/set_printoptions with a context manager accessed using numpy.printoptions in the same way that numpy.errstate exposes a context manager to seterr/geterr? This would make the set method redundant. Also, the context manager returned by numpy.errstate, numpy.printoptions, etc. could expose the dictionary directly. This would make the get methods redundant. Best, Neil

3 2

diferences between frompyfunc & vectorize
by Tim Michelsen 27 Oct '13

27 Oct '13

Hello, what are the differences between the two transformation functions frompyfunc & vectorize * http://docs.scipy.org/doc/numpy/reference/generated/numpy.vectorize.html * http://docs.scipy.org/doc/numpy/reference/generated/numpy.frompyfunc.html I cannot see the which one I shall prefer for what purpose. Thanks in advance, Timmie

1 0

vectorizing recursive sequences
by Jaime Fernández del Río 27 Oct '13

27 Oct '13

I recently came up with a way of vectorizing some recursive sequence calculations. While it works, I am afraid it is relying on implementation details potentially subject to change. The basic idea is illustrated by this function, calculating the first n items of the Fibonacci sequence: def fibonacci(n): fib = np.empty(n, dtype=np.intp) fib[:2] = 1 np.add(fib[:-2], fib[1:-1], out=fib[2:]) return fib >>> fibonacci(10) array([ 1, 1, 2, 3, 5, 8, 13, 21, 34, 55], dtype=int64) I believe that the biggest issue that could break this is if the ufunc decided to buffer the arrays, as this is relying on the inputs and outputs of np.add sharing the same memory. You can use the same idea to do more complicated things, for instance to calculate the items of the sequence: f[0] = a[0] f[n] = a[n] + x * f[n-1] from numpy.lib.stride_tricks import as_strided from numpy.core.umath_tests import inner1d def f(a, x): out = np.array(a, copy=True, dtype=np.double) n = len(a) out_view = as_strided(out, shape=(n-1, 2), strides=out.strides*2) inner1d(out_view, [x, 1], out=out[1:]) return out >>> f(np.arange(10), 0.1) array([ 0. , 1. , 2.1 , 3.21 , 4.321 , 5.4321 , 6.54321 , 7.654321 , 8.7654321 , 9.87654321]) Again, I think buffering is the clearest danger of doing something like this, as the first input and output must share the same memory for this to work. That this is a real concern is easy to see: since `inner1d` only has loops registered for long ints and double floats: >>> inner1d.types ['ll->l', 'dd->d'] the above function `f` doesn't work if the `out` array is created, e.g. as np.float32, since there will be buffering happening because of the type casting. So I have two questions: 1. Is there some other reason, aside from buffering, that could go wrong, or change in a future release? 2. As far as buffering is concerned, I thought of calling `np.setbuffer(1)` before any of these functions, but it complains and requests that the value be a multiple of 16. Is there any other way to ensure that the data is fetched from an updated array in every internal iteration? Thanks! Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

3 2

Re: [Numpy-discussion] [Pythonmac-SIG] Building extension modules with Xcode 5
by Chris Barker 26 Oct '13

26 Oct '13

Ned, I think this fell off the list (I really don't like when reply is not set to the list...) On Fri, Oct 25, 2013 at 4:33 PM, Ned Deily <nad(a)acm.org> wrote: > > On Oct 25, 2013, at 15:17 , Chris Barker <chris.barker(a)noaa.gov> wrote: > > > On Thu, Oct 24, 2013 at 7:29 PM, Ned Deily <nad(a)acm.org> wrote: > > I'm really confused here. Using the current 2.7.5 64-bit installer > (from > > May) on 10.8.5 with Xcode 5.0.1 works just fine for me. > > > > I'm not on 10.8 or XCode5, but does it make a difference whether you > installed XCode 5 after Xcode 4? i.e. someone may have gcc4.2 around form > XCode 4, but not if they installed XCode 5 on a raw machine. > > Not that I know of. gcc-4.2 is a red herring as far as I can tell. > Distutils has known how to deal with a missing compiler for a couple of > releases now. Good to know -- I know I really need to move forward -- I wonder if there is an XCode5 for 10.6? pProbably not -- but for some screwy reason Apple does not provide a free XCode4 for 10.6, either (though you can hack the installer for 10.7 ... > However, I did just realize that I had forgotten about a problem when > using the current 32-bit-only installers with Xcode 5. I've just posted > about that; it's described now in http://bugs.python.org/issue19400 > > Good to fix that, but does anyone need it anymore? Though I use it when I need to force 32 bit, even on a 64 bit system (I'm working with some not-64-bit safe C++ code...) -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

2 1

Re: [Numpy-discussion] NumPy 1.8.0rc2 release
by Andrew Straw 25 Oct '13

25 Oct '13

Hi, I found an unexpected difference from numpy 1.7.1 and 1.8.0rc2 with Python 3.3.2 on Ubuntu 12.04 (amd64). Here is the test program: import numpy as np print(np.__version__) K = np.array([[ 0. , 0. , 0. , 0. ], [-0.33333333, 0. , 0. , 0. ], [ 0.33333333, -0.33333333, 0. , 0. ], [ 0.33333333, -0.33333333, 0.33333333, 0. ]]) w, V = np.linalg.eigh(K) print('w') print(w) with numpy 1.7.1: 1.7.1 w [-0.33333333 -0.33333333 -0.33333333 0.99999999] with numpy 1.8.0rc2: 1.8.0rc2 w [ 0. 0. 0. 0.] Apologies if this is my mistake! -Andrew

4 4

ANN: SciPy 0.13.0 release
by Ralf Gommers 25 Oct '13

25 Oct '13

On behalf of the SciPy development team I'm pleased to announce the availability of SciPy 0.13.0. This release contains some interesting new features (see highlights below) and half a year's worth of maintenance work. 65 people contributed to this release. Some of the highlights are: - support for fancy indexing and boolean comparisons with sparse matrices - interpolative decompositions and matrix functions in the linalg module - two new trust-region solvers for unconstrained minimization This release requires Python 2.6, 2.7 or 3.1-3.3 and NumPy 1.5.1 or greater. Support for Python 2.4 and 2.5 has been dropped as of this release. Sources and binaries can be found at http://sourceforge.net/projects/scipy/files/scipy/0.13.0/, release notes are copied below. Enjoy, Ralf ========================== SciPy 0.13.0 Release Notes ========================== .. contents:: SciPy 0.13.0 is the culmination of 7 months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Moreover, our development attention will now shift to bug-fix releases on the 0.13.x branch, and on adding new features on the master branch. This release requires Python 2.6, 2.7 or 3.1-3.3 and NumPy 1.5.1 or greater. Highlights of this release are: - support for fancy indexing and boolean comparisons with sparse matrices - interpolative decompositions and matrix functions in the linalg module - two new trust-region solvers for unconstrained minimization New features ============ ``scipy.integrate`` improvements -------------------------------- N-dimensional numerical integration ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ A new function `scipy.integrate.nquad`, which provides N-dimensional integration functionality with a more flexible interface than ``dblquad`` and ``tplquad``, has been added. ``dopri*`` improvements ^^^^^^^^^^^^^^^^^^^^^^^ The intermediate results from the ``dopri`` family of ODE solvers can now be accessed by a *solout* callback function. ``scipy.linalg`` improvements ----------------------------- Interpolative decompositions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ Scipy now includes a new module `scipy.linalg.interpolative` containing routines for computing interpolative matrix decompositions (ID). This feature is based on the ID software package by P.G. Martinsson, V. Rokhlin, Y. Shkolnisky, and M. Tygert, previously adapted for Python in the PymatrixId package by K.L. Ho. Polar decomposition ^^^^^^^^^^^^^^^^^^^ A new function `scipy.linalg.polar`, to compute the polar decomposition of a matrix, was added. BLAS level 3 functions ^^^^^^^^^^^^^^^^^^^^^^ The BLAS functions ``symm``, ``syrk``, ``syr2k``, ``hemm``, ``herk`` and ``her2k`` are now wrapped in `scipy.linalg`. Matrix functions ^^^^^^^^^^^^^^^^ Several matrix function algorithms have been implemented or updated following detailed descriptions in recent papers of Nick Higham and his co-authors. These include the matrix square root (``sqrtm``), the matrix logarithm (``logm``), the matrix exponential (``expm``) and its Frechet derivative (``expm_frechet``), and fractional matrix powers (``fractional_matrix_power``). ``scipy.optimize`` improvements ------------------------------- Trust-region unconstrained minimization algorithms ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The ``minimize`` function gained two trust-region solvers for unconstrained minimization: ``dogleg`` and ``trust-ncg``. ``scipy.sparse`` improvements ----------------------------- Boolean comparisons and sparse matrices ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ All sparse matrix types now support boolean data, and boolean operations. Two sparse matrices `A` and `B` can be compared in all the expected ways `A < B`, `A >= B`, `A != B`, producing similar results as dense Numpy arrays. Comparisons with dense matrices and scalars are also supported. CSR and CSC fancy indexing ^^^^^^^^^^^^^^^^^^^^^^^^^^ Compressed sparse row and column sparse matrix types now support fancy indexing with boolean matrices, slices, and lists. So where A is a (CSC or CSR) sparse matrix, you can do things like:: >>> A[A > 0.5] = 1 # since Boolean sparse matrices work >>> A[:2, :3] = 2 >>> A[[1,2], 2] = 3 ``scipy.sparse.linalg`` improvements ------------------------------------ The new function ``onenormest`` provides a lower bound of the 1-norm of a linear operator and has been implemented according to Higham and Tisseur (2000). This function is not only useful for sparse matrices, but can also be used to estimate the norm of products or powers of dense matrices without explictly building the intermediate matrix. The multiplicative action of the matrix exponential of a linear operator (``expm_multiply``) has been implemented following the description in Al-Mohy and Higham (2011). Abstract linear operators (`scipy.sparse.linalg.LinearOperator`) can now be multiplied, added to each other, and exponentiated, producing new linear operators. This enables easier construction of composite linear operations. ``scipy.spatial`` improvements ------------------------------ The vertices of a `ConvexHull` can now be accessed via the `vertices` attribute, which gives proper orientation in 2-D. ``scipy.signal`` improvements ----------------------------- The cosine window function `scipy.signal.cosine` was added. ``scipy.special`` improvements ------------------------------ New functions `scipy.special.xlogy` and `scipy.special.xlog1py` were added. These functions can simplify and speed up code that has to calculate ``x * log(y)`` and give 0 when ``x == 0``. ``scipy.io`` improvements ------------------------- Unformatted Fortran file reader ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The new class `scipy.io.FortranFile` facilitates reading unformatted sequential files written by Fortran code. ``scipy.io.wavfile`` enhancements ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `scipy.io.wavfile.write` now accepts a file buffer. Previously it only accepted a filename. `scipy.io.wavfile.read` and `scipy.io.wavfile.write` can now handle floating point WAV files. ``scipy.interpolate`` improvements ---------------------------------- B-spline derivatives and antiderivatives ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `scipy.interpolate.splder` and `scipy.interpolate.splantider` functions for computing B-splines that represent derivatives and antiderivatives of B-splines were added. These functions are also available in the class-based FITPACK interface as ``UnivariateSpline.derivative`` and ``UnivariateSpline.antiderivative``. ``scipy.stats`` improvements ---------------------------- Distributions now allow using keyword parameters in addition to positional parameters in all methods. The function `scipy.stats.power_divergence` has been added for the Cressie-Read power divergence statistic and goodness of fit test. Included in this family of statistics is the "G-test" (http://en.wikipedia.org/wiki/G-test). `scipy.stats.mood` now accepts multidimensional input. An option was added to `scipy.stats.wilcoxon` for continuity correction. `scipy.stats.chisquare` now has an `axis` argument. `scipy.stats.mstats.chisquare` now has `axis` and `ddof` arguments. Deprecated features =================== ``expm2`` and ``expm3`` ----------------------- The matrix exponential functions `scipy.linalg.expm2` and `scipy.linalg.expm3` are deprecated. All users should use the numerically more robust `scipy.linalg.expm` function instead. ``scipy.stats`` functions ------------------------- `scipy.stats.oneway` is deprecated; `scipy.stats.f_oneway` should be used instead. `scipy.stats.glm` is deprecated. `scipy.stats.ttest_ind` is an equivalent function; more full-featured general (and generalized) linear model implementations can be found in statsmodels. `scipy.stats.cmedian` is deprecated; ``numpy.median`` should be used instead. Backwards incompatible changes ============================== LIL matrix assignment --------------------- Assigning values to LIL matrices with two index arrays now works similarly as assigning into ndarrays:: >>> x = lil_matrix((3, 3)) >>> x[[0,1,2],[0,1,2]]=[0,1,2] >>> x.todense() matrix([[ 0., 0., 0.], [ 0., 1., 0.], [ 0., 0., 2.]]) rather than giving the result:: >>> x.todense() matrix([[ 0., 1., 2.], [ 0., 1., 2.], [ 0., 1., 2.]]) Users relying on the previous behavior will need to revisit their code. The previous behavior is obtained by ``x[numpy.ix_([0,1,2],[0,1,2])] = ...`. Deprecated ``radon`` function removed ------------------------------------- The ``misc.radon`` function, which was deprecated in scipy 0.11.0, has been removed. Users can find a more full-featured ``radon`` function in scikit-image. Removed deprecated keywords ``xa`` and ``xb`` from ``stats.distributions`` -------------------------------------------------------------------------- The keywords ``xa`` and ``xb``, which were deprecated since 0.11.0, have been removed from the distributions in ``scipy.stats``. Changes to MATLAB file readers / writers ---------------------------------------- The major change is that 1D arrays in numpy now become row vectors (shape 1, N) when saved to a MATLAB 5 format file. Previously 1D arrays saved as column vectors (N, 1). This is to harmonize the behavior of writing MATLAB 4 and 5 formats, and adapt to the defaults of numpy and MATLAB - for example ``np.atleast_2d`` returns 1D arrays as row vectors. Trying to save arrays of greater than 2 dimensions in MATLAB 4 format now raises an error instead of silently reshaping the array as 2D. ``scipy.io.loadmat('afile')`` used to look for `afile` on the Python system path (``sys.path``); now ``loadmat`` only looks in the current directory for a relative path filename. Other changes ============= Security fix: ``scipy.weave`` previously used temporary directories in an insecure manner under certain circumstances. Cython is now required to build *unreleased* versions of scipy. The C files generated from Cython sources are not included in the git repo anymore. They are however still shipped in source releases. The code base received a fairly large PEP8 cleanup. A ``tox pep8`` command has been added; new code should pass this test command. Scipy cannot be compiled with gfortran 4.1 anymore (at least on RH5), likely due to that compiler version not supporting entry constructs well. Authors ======= This release contains work by the following people (contributed at least one patch to this release, names in alphabetical order): * Jorge Cañardo Alastuey + * Tom Aldcroft + * Max Bolingbroke + * Joseph Jon Booker + * François Boulogne * Matthew Brett * Christian Brodbeck + * Per Brodtkorb + * Christian Brueffer + * Lars Buitinck * Evgeni Burovski + * Tim Cera * Lawrence Chan + * David Cournapeau * Dražen Lučanin + * Alexander J. Dunlap + * endolith * André Gaul + * Christoph Gohlke * Ralf Gommers * Alex Griffing + * Blake Griffith + * Charles Harris * Bob Helmbold + * Andreas Hilboll * Kat Huang + * Oleksandr (Sasha) Huziy + * Gert-Ludwig Ingold + * Thouis (Ray) Jones * Juan Luis Cano Rodríguez + * Robert Kern * Andreas Kloeckner + * Sytse Knypstra + * Gustav Larsson + * Denis Laxalde * Christopher Lee * Tim Leslie * Wendy Liu + * Clemens Novak + * Takuya Oshima + * Josef Perktold * Illia Polosukhin + * Przemek Porebski + * Steve Richardson + * Branden Rolston + * Skipper Seabold * Fazlul Shahriar * Leo Singer + * Rohit Sivaprasad + * Daniel B. Smith + * Julian Taylor * Louis Thibault + * Tomas Tomecek + * John Travers * Richard Tsai + * Jacob Vanderplas * Patrick Varilly * Pauli Virtanen * Stefan van der Walt * Warren Weckesser * Pedro Werneck + * Nils Werner + * Michael Wimmer + * Nathan Woods + * Tony S. Yu + A total of 65 people contributed to this release. People with a "+" by their names contributed a patch for the first time.

4 3

"official" binaries on web page.
by Chris Barker 23 Oct '13

23 Oct '13

If you go to numpy.org, and try to figure out how to install numpy, you are most likely to end up here: http://www.scipy.org/install.html where there is no mention of the binaries built by the numpy project itself, either Windows or Mac. There probably should be. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker(a)noaa.gov

8 27

profiling numpy functions
by alex 23 Oct '13

23 Oct '13

I have a question about numpy profiling. When I run my script with -m cProfile I see timings for lots of functions, but not for some like np.exp(). How do I see these timings? I don't want to have to work around it by writing a lot of functions like def mynumpyexp(A): return np.exp(A).

1 0