Mailman 3 November 2014 - NumPy-Discussion

return type from ufuncs
by Marek Wojciechowski Nov. 27, 2014

Nov. 27, 2014

Hi! I wrote a simple subclass of np.ndarray and now i do call np.sum() on it. I expected that the result will be a python float (or int) just like when summing up regular arrays. Instead i obtain the (scalar) view of my subclass. How can i change this behavior? I tried writing __array_wrap__ method like this: def __array_wrap__(self, out_arr, context=None): selfv = self.view(np.ndarray) return np.ndarray.__array_wrap__(selfv, out_arr, context) but this just returns np.ndarray type and not float. Regards, -- Marek Wojciechowski

2 3

ANN: Scipy 0.15.0 beta 1 release
by Pauli Virtanen Nov. 26, 2014

Nov. 26, 2014

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Dear all, We have finally finished preparing the Scipy 0.15.0 beta 1 release. Please try it and report any issues on the scipy-dev mailing list, and/or on Github. If no surprises turn up, the final release is planned on Dec 20 in three weeks. Source tarballs and full release notes are available at https://sourceforge.net/projects/scipy/files/SciPy/0.15.0b1/ Binary installers should also be up soon. Best regards, Pauli Virtanen - -------------------------------------------- SciPy 0.15.0 is the culmination of 6 months of hard work. It contains several new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Moreover, our development attention will now shift to bug-fix releases on the 0.16.x branch, and on adding new features on the master branch. This release requires Python 2.6, 2.7 or 3.2-3.3 and NumPy 1.5.1 or greater. New features ============ Linear Programming Interface - - ---------------------------- The new function ``scipy.optimize.linprog`` provides a generic linear programming similar to the way ``scipy.optimize.minimize`` provides a generic interface to nonlinear programming optimizers. Currently the only method supported is *simplex* which provides a two-phase, dense-matrix-based simplex algorithm. Callbacks functions are supported,allowing the user to monitor the progress of the algorithm. Differential_evolution, a global optimizer - - ------------------------------------------ A new ``differential_evolution`` function is available in the ``scipy.optimize`` module. Differential Evolution is an algorithm used for finding the global minimum of multivariate functions. It is stochastic in nature (does not use gradient methods), and can search large areas of candidate space, but often requires larger numbers of function evaluations than conventional gradient based techniques. ``scipy.signal`` improvements - - ----------------------------- The function ``max_len_seq`` was added, which computes a Maximum Length Sequence (MLS) signal. ``scipy.integrate`` improvements - - -------------------------------- It is now possible to use ``scipy.integrate`` routines to integrate multivariate ctypes functions, thus avoiding callbacks to Python and providing better performance. ``scipy.linalg`` improvements - - ----------------------------- Add function ``orthogonal_procrustes`` for solving the procrustes linear algebra problem. ``scipy.sparse`` improvements - - ----------------------------- ``scipy.sparse.linalg.svds`` can now take a ``LinearOperator`` as its main input. ``scipy.special`` improvements - - ------------------------------ Values of ellipsoidal harmonic (i.e. Lame) functions and associated normalization constants can be now computed using ``ellip_harm``, ``ellip_harm_2``, and ``ellip_normal``. New convenience functions ``entr``, ``rel_entr`` ``kl_div``, ``huber``, and ``pseudo_huber`` were added. ``scipy.sparse.csgraph`` improvements - - ------------------------------------- Routines ``reverse_cuthill_mckee`` and ``maximum_bipartite_matching`` for computing reorderings of sparse graphs were added. ``scipy.stats`` improvements - - ---------------------------- Added a Dirichlet distribution as multivariate distribution. The new function ``scipy.stats.median_test`` computes Mood's median test. The new function ``scipy.stats.combine_pvalues`` implements Fisher's and Stouffer's methods for combining p-values. ``scipy.stats.describe`` returns a namedtuple rather than a tuple, allowing users to access results by index or by name. Deprecated features =================== The ``scipy.weave`` module is deprecated. It was the only module never ported to Python 3.x, and is not recommended to be used for new code - use Cython instead. In order to support existing code, ``scipy.weave`` has been packaged separately: `https://github.com/scipy/weave`_. It is a pure Python package, and can easily be installed with ``pip install weave``. ``scipy.special.bessel_diff_formula`` is deprecated. It is a private function, and therefore will be removed from the public API in a following release. Backwards incompatible changes ============================== scipy.ndimage - - ------------- The functions ``scipy.ndimage.minimum_positions``, ``scipy.ndimage.maximum_positions`` and ``scipy.ndimage.extrema`` return positions as ints instead of floats. scipy.integrate - - --------------- The format of banded Jacobians in ``scipy.integrate.ode`` solvers is changed. Note that the previous documentation of this feature was erroneous. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1 iEYEARECAAYFAlRyaf8ACgkQ6BQxb7O0pWC7XQCeNtdJD4ZNDXvFeNFs7N3KjQn6 8QkAoK3pFmhMrTwCrgkusl+fRNMboN2r =WSpM -----END PGP SIGNATURE-----

8 13

Implementation of tensor product and tensor contraction?
by Yuxiang Wang Nov. 26, 2014

Nov. 26, 2014

Dear all, I have been doing tensor algebra recently (for continuum mechanics) and was looking into two common operations: tensor product & tensor contraction. 1. Tensor product One common usage is: a[i1, i2, i3, ..., iN, j1, j2, j3, ..., jM] = b[i1, i2, i3, ..., iN] * c[j1, j2, j3, ..., jM] I looked into the current np.outer(), and the only difference is that it always flattens the input array. So actually, the function for tensor product is simply np.outer(a, b, out=out).reshape(a.shape+b.shape) <-- I think I got this right but please do correct me if I am wrong Would anyone think it helpful or harmful to add such a function, np.tensorprod()? it will simply be like def tensorprod(a, b, out=None): return outer(a, b, out=out).reshape(a.shape+b.shape) 2. Tensor contraction It is currently the np.tensordot(a, b) and it will do np.tensordot(a, b, axes=2) by default. I think this is all great, but it would be even better if we specify in the doc, that: i) say explicitly that by default it will be the double-dot or contraction operator, and ii) explain that in cases where axes is an integer-like scalar, which axes were selected from the two array and in what order. Like: if axes is an integer-like scalar, it is the number axes to sum over, equivalent to axes=(list(range(-axes, 0)), list(range(0, axes))) (or something like this) It'd be great to hear what you would think about it, Shawn -- Yuxiang "Shawn" Wang Gerling Research Lab University of Virginia yw5aj(a)virginia.edu +1 (434) 284-0836 https://sites.google.com/a/virginia.edu/yw5aj/

2 2

Re: [Numpy-discussion] [SciPy-Dev] ANN: Scipy 0.15.0 beta 1 release
by Nathaniel Smith Nov. 25, 2014

Nov. 25, 2014

On Tue, Nov 25, 2014 at 7:07 PM, Sturla Molden <sturla.molden(a)gmail.com> wrote: > Nathaniel Smith <njs(a)pobox.com> wrote: >> On Tue, Nov 25, 2014 at 12:53 PM, David Cournapeau <cournape(a)gmail.com> wrote: >>> Shall we consider https://github.com/scipy/scipy/issues/4168 to be a blocker >>> (the issue arises on scipy master as well as 0.14.1) ? >> >> Do you think there's anything scipy can do about it? It looks like a >> pure numpy bug to me. > > I think we should assume it is not a bug in NumPy. There is nothing that > dictates that an ndarray's buffer must be aligned. It can just as well be > an unaligned view or wrap an external buffer. An ndarray is valud with any > alignment, but it does not mean it can be used with intent(inout) in f2py. > > Then by deduction if it is not a NumPy bug it must be a SciPy bug, which > means the bug must be the unprotected use of intent(inout) in the f2py > wrapper. I would think it is the responsibility of SciPy to ensure that > whatever is passed with intent(inout) in SciPy is properly aligned. In general you have a point, but in this case the reason the array is "unaligned" is that numpy is being overly strict about its definition of "aligned". In fact, it's so strict that numpy itself doesn't provide any standard and reliable way to *create* an aligned array by this definition. (I guess scipy could create an overallocated copy and then take a slice at the right offset, but asking scipy to use such hacks to work around our bugs is clearly wrong.) -n -- Nathaniel J. Smith Postdoctoral researcher - Informatics - University of Edinburgh http://vorpus.org

2 1

Initializing array from buffer
by Andrea Arteaga Nov. 23, 2014

Nov. 23, 2014

Hello. Using the numpy.frombuffer function [1] one can initialize a numpy array using an existing python object that implements the buffer protocol [2]. This is great, but currently this function supports only 1D buffers, even if the provided buffer is multidimensional and it exposes all information about its structure (shape, strides, data type). Apparently, one can extract every kind of buffer information out of a buffer of a numpy array (pointer, number of dimensions, shape, strides, suboffsets,...), but the other way around is only partially implemented: providing a multidimensional buffer does not mean being able of creating a numpy array the uses that buffer with the desired structure. My use case is the following: we have a some 3D arrays in our C++ framework. The ordering of the elements in these arrays is neither C nor Fortran style: it might be IJK (i.e. C style, 3rd dimension contiguous in memory), KJI (i.e. Fortran style, first dimension contiguous) or, e.g. IKJ. Moreover we put some padding to optimize aligned access. This kind of memory structure cannot be just expressed as 'C' or 'Fortran', but it can be perfectly expressed using the Python buffer protocol by providing the shape and the strides. We would like to export this structure to a numpy array that should be able of accessing the same memory locations in a consistent way and make some operations like initializing the content or plotting it. Is this currently possible? If not, is it planned to implement such a feature? ========== Maybe just to clarify I could show an example entirely in python. Assume a in a 2D numpy array: a = np.ones((10,20)) It contains information about its structure which can be portably accessed using its data member: a.data.format == 'd' a.data.ndim == 2 a.data.shape == (10,20) a.data.strides == (160,8) Unfortunately, when initializing an array b from this buffer, the structure of the buffer is "downgraded" to unidimensional shape: b = np.frombuffer(a.data) b.ndim == 1 b.shape == (200,) b.strides == (8,) I wished b had the same multi-dimensional structure of a. (This is of course a very simple example. In my use case I would initialize b with my own buffer instead of that of another numpy array). Best regards [1] http://docs.scipy.org/doc/numpy/reference/generated/numpy.frombuffer.html [2] https://docs.python.org/3/c-api/buffer.html

6 8

ANN: First release of combi, a combinatorics package for Python
by Ram Rachum Nov. 22, 2014

Nov. 22, 2014

Hi everyone, Just a quick announcement: Last week I made the first release of an open-source package called Combi. It's a package for doing combinatorics in Python. The pitch: Combi lets you explore spaces of permutations and combinations as if they were Python sequences, but without generating all the permutations/combinations in advance. It lets you specify a lot of special conditions on these spaces. It also provides a few more classes that might be useful in combinatorics programming. Information about it: https://pypi.python.org/pypi/combi I hope this package will be useful to people here. Thanks, Ram.

1 0

Numpy 1.9.1, zeros and alignement
by David Cournapeau Nov. 20, 2014

Nov. 20, 2014

Hi, I have not followed closely the changes that happen in 1.9.1, but was surprised by the following: x = np.zeros(12, "d") assert x.flags.aligned # fails This is running numpy 1.9.1 built on windows with VS 2008. Is it expected that zeros may return a non-aligned array ? David

5 10

truthiness of object arrays
by Antony Lee Nov. 14, 2014

Nov. 14, 2014

I am puzzled by the following (numpy 1.9.0, python 3.4.2): In [1]: t = array(None); t[()] = array([None, None]) # Construct a 0d array of dtype object, containing a single numpy array with 2 elements In [2]: bool(t) Out[2]: True In [3]: if t: pass --------------------------------------------------------------------------- ValueError Traceback (most recent call last) ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all() I thought that "if x" simply calls "bool", but apparently this is not even the case... Antony

6 11

Re: [Numpy-discussion] #2522 numpy.diff fails on unsigned integers
by Sebastian Nov. 13, 2014

Nov. 13, 2014

On 2014-11-04 19:44, Charles R Harris wrote: > On Tue, Nov 4, 2014 at 11:19 AM, Sebastian <sebix(a)sebix.at> wrote: > >> On 2014-11-04 15:06, Todd wrote: >>> On Tue, Nov 4, 2014 at 2:50 PM, Sebastian Wagner <sebix(a)sebix.at >> >>> <mailto:sebix@sebix.at>> wrote: >>> >>> Hello, >>> >>> I want to bring up Issue #2522 'numpy.diff fails on unsigned >> integers >>> (Trac #1929)' [1], as it was resonsible for an error in one >> of our >>> programs. Short explanation of the bug: np.diff performs a >> subtraction >>> on the input array. If this is of type uint and the data >> contains >>> falling data, it results in an artihmetic underflow. >>> >>> >>> np.diff(np.array([0,1,0], dtype=np.uint8)) >>> array([ 1, 255], dtype=uint8) >>> >>> @charris proposed either >>> - a note to the doc string and maybe an example to clarify >> things >>> - or raise a warning >>> but with a discussion on the list. >>> >>> I would like to start it now, as it is an error which is not >> easily >>> detectable (no errors or warnings are thrown). In our case >> the >>> type of a >>> data sequence, with only zeros and ones, had type f8 as also >> every >>> other >>> one, has been changed to u4. As the programs looked for >> values ==1 and >>> ==-1, it broke silently. >>> In my opinion, a note in the docs is not enough and does not >> help >>> if the >>> type changed or set after the program has been written. >>> I'd go for automatic upcasting of uints by default and an >> option >>> to turn >>> it off, if this behavior is explicitly wanted. This wouldn't >> be >>> correct >>> from the point of view of a programmer, but as most of the >> users >>> have a >>> scientific background who excpect it 'to work', instead of >> sth is >>> theoretically correct but not convenient. (I count myself to >> the first >>> group) >>> >>> >>> >>> When you say "automatic upcasting", that would be, for example >> uint8 >>> to int16? What about for uint64? There is no int128. >> The upcast should go to the next bigger, otherwise it would again >> result >> in wrong values. uint64 we can't do that, so it has to stay. >>> Also, when you say "by default", is this only when an overflow is >>> detected, or always? >> I don't know how I could detect an overflow in the diff-function. >> In >> subtraction it should be possible, but that's very deep in the >> numpy-internals. >>> How would the option to turn it off be implemented? An argument >> to >>> np.diff or some sort of global option? >> I thought of a parameter upcast_int=True for the function. > > Could check for non-decreasing sequence in the unsigned case. Note > that differences of signed integers can also overflow. One way to > check in general is to determine the expected sign using comparisons. I think you mean a decreasing/non-increasing instead of non-decreasing sequence? It's also the same check as checking for a sorted sequence. But I currently don't know how I could do that efficiently without np.diff in Python, in Cython it should be easily possible. np.gradient has the same problem: >>> np.random.seed(89) >>> d = np.random.randint(0,2,size=10).astype(np.uint8); d array([1, 0, 0, 1, 0, 1, 1, 0, 0, 0], dtype=uint8) >>> np.diff(d) array([255, 0, 1, 255, 1, 0, 255, 0, 0], dtype=uint8) >>> np.gradient(d) array([ 255. , 127.5, 0.5, 0. , 0. , 0.5, 127.5, 127.5, 0. , 0. ]) --- gpg --keyserver keys.gnupg.net --recv-key DC9B463B

3 2

Subscribing to mailinglist not possible / sites down
by Stefan Otte Nov. 13, 2014

Nov. 13, 2014

Hey *, The websites to subscribe to the numpy/scipy mailinglists seem to be down: - http://mail.scipy.org/mailman/listinfo/scipy-user - http://mail.scipy.org/mailman/listinfo/scipy-user - http://projects.scipy.org/pipermail/scipy-dev/ And it's not just me: http://www.downforeveryoneorjustme.com/http://projects.scipy.org/pipermail/… Best, Stefan

2 1