Mailman 3 June 2016 - NumPy-Discussion

Axis argument to np.unique
by Martino Sorbaro June 15, 2016

June 15, 2016

Hi all, I've opened a new pull request (https://github.com/numpy/numpy/pull/7742) trying to revive a previous one that was left abandoned (#3584, by another contributor), regarding the possibility of adding an 'axis=' argument to numpy.unique. There had been a debate (http://numpy-discussion.10968.n7.nabble.com/Adding-an-axis-argument-to-nump…) about what the axis argument should mean. The current behaviour in the code I propose (written by the previous contributor) looks for unique rows if "axis=0" and unique columns if "axis=1", in other words: [In] a = array([[0, 0, 0], [1, 1, 1], [0, 0, 0]]) [In] unique(a, axis=0) [Out] array([[0, 0, 0], [1, 1, 1]]) So, I would just like to ask whether a conclusion can be reached about that discussion. Thanks! Martino -- The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336.

2 1

scipy.stats.qqplot and scipy.stats.probplot axis labeling
by Mark Gawron June 12, 2016

June 12, 2016

The scipy.stats.qqplot and scipy.stats.probplot functions plot expected values versus actual data values for visualization of fit to a distribution. First a one-D array of expected percentiles is generated for a sample of size N; then that is passed to dist.ppf, the per cent point function for the chosen distribution, to return an array of expected values. The visualized data points are pairs of expected and actual values, and a linear regression is done on these to produce the line data points in this distribution should lie on. Where x is the input data array and dist the chosen distribution we have: > osr = np.sort(x) > osm_uniform = _calc_uniform_order_statistic_medians(len(x)) > osm = dist.ppf(osm_uniform) > slope, intercept, r, prob, sterrest = stats.linregress(osm, osr) My question concerns the plot display. > ax.plot(osm, osr, 'bo', osm, slope*osm + intercept, 'r-') The x-axis of the resulting plot is labeled quantiles, but the xticks and xticklabels produced produced by qqplot and problplot do not seem correct for the their intended interpretations. First the numbers on the x-axis do not represent quantiles; the intervals between them do not in general contain equal numbers of points. For a normal distribution with sigma=1, they represent standard deviations. Changing the label on the x-axis does not seem like a very good solution, because the interpretation of the values on the x-axis will be different for different distributions. Rather the right solution seems to be to actually show quantiles on the x-axis. The numbers on the x-axis can stay as they are, representing quantile indexes, but they need to be spaced so as to show the actual division points that carve the population up into groups of the same size. This can be done in something like the following way. > import numpy as np > xt = np.arange(-3,3,dtype=int) > # Find the 5 quantiles to divide the data into sixths > percentiles = [x*.167 + .502 for x in xt] > percentiles = np.array(percentiles + [.999]) > vals = dist.ppf(percentiles) > ax.set_xticks(vals) > xt = np.array(list(xt)+[3]) > ax.set_xticklabels(xt) > ax.set_xlabel('Quantile') > plt.show() I’ve attached two images to show the difference between the current visualization and the suggested one. Mark Gawron

3 6

NumPy lesson at EuroScipy2016?
by mail＠telenczuk.pl June 11, 2016

June 11, 2016

Hi all, Recently I taught "Advanced NumPy" lesson at a Software Carpentry workshop [1]. It covered a review of basic operations on numpy arrays and also more advanced topics: indexing, broadcasting, dtypes and memory layout. I would greatly appreciate your feedback on the lesson materials, which are available on github pages [2]. I am also thinking of proposing this lesson as a EuroScipy 2016 tutorial. Is anyone already planning to teach NumPy there? If so, would you be interested to team up for this lesson (as a co-instructor, helper or mentor)? I gratefully acknowledge inspiration, some examples and exercises from the following materials: - NumPy chapters of "SciPy lectures" by Emmanuelle Gouillart, Didrik Pinte, Gaël Varoquaux, and Pauli Virtanen [3] - "Advanced NumPy patterns" by Juan Nunez-Iglesias [4] - "The NumPy array. A structure for efficient numerical computation." by Stefan van der Walt [5] Yours, Bartosz [1] http://telecom-python.telenczuk.pl [2] https://paris-swc.github.io/advanced-numpy-lesson/ [3] http://www.scipy-lectures.org/ [4] https://github.com/jni/aspp2015/tree/delivered [5] https://python.g-node.org/python-summerschool-2014/numpy.html

3 2

Datarray 0.1.0 release
by Matthew Brett June 10, 2016

June 10, 2016

Hi, I just released a new version of the Datarray package: https://pypi.python.org/pypi/datarray/0.1.0 https://github.com/BIDS/datarray It's a very lightweight implementation of arrays with labeled axes and ticks, that allows you to do stuff like: >>> narr = DataArray(np.zeros((1,2,3)), axes=('a','b','c')) >>> narr.axes.a Axis(name='a', index=0, labels=None) >>> narr.axes.a[0] DataArray(array([[ 0., 0., 0.], [ 0., 0., 0.]]), ('b', 'c')) It's still experimental, but we'd love to hear any feedback, in any form. Please feel free to make github issues for specific bugs / suggestions: https://github.com/BIDS/datarray/issues If you like the general idea, and you don't mind the pandas dependency, `xray` is a much better choice for production code right now, and will do the same stuff and more: https://pypi.python.org/pypi/xray/0.4.1 Cheers, Matthew

2 2

Indexing with floats
by Fabien June 10, 2016

June 10, 2016

Hi, I really tried to do my homework before asking this here, but I just couldn't find the relevant information anywhere... My question is about the rationale behind forbidding indexing with floats, i.e.: >>> x[2.] __main__:1: VisibleDeprecationWarning: using a non-integer number instead of an integer will result in an error in the future I don't find this very handy from a user's perspective, and I'd be grateful for pointers on discussion threads and/or PRs where this has been discussed, so that I can understand why it's important. Maybe a short note on the indexing docpage (http://docs.scipy.org/doc/numpy-1.11.0/user/basics.indexing.html) could be useful also. Thanks a lot! Fabien

2 2

Requesting a PR review for #5822
by Antony Lee June 9, 2016

June 9, 2016

https://github.com/numpy/numpy/pull/5822 is a year-old PR which allows many random distributions to have a scale of exactly 0 (in which case a stream of zeros is returned of whatever constant value is appropriate). It passes all tests and has been sitting there for a while. Would a core dev be kind enough to have a look at it? Thanks! Antony

1 0

EuroSciPy 2016
by Pierre de Buyl June 7, 2016

June 7, 2016

Dear NumPy and SciPy communities, No annoucement was made here for EuroSciPy 2016 I believe. The call for contributions (talks, posters, sprints) is still open for a few days. EuroSciPy 2016 takes place in Erlangen, Germany, from the 23 to the 27 of August and consists of two days of tutorials (beginner and advanced tracks) and two days of conference representing many fields of science, with a focus on Python tools for science. A day of sprints follows (sprints TBA). The keynote speakers are Gaël Varoquaux and Abby Cabunoc Mayes and we can expect a rich tutorial and scientific program! Videos from previous years are available at https://www.youtube.com/playlist?list=PLYx7XA2nY5GeQCCugyvtnHMVLdhYlrRxH and https://www.youtube.com/playlist?list=PLYx7XA2nY5Gcpabmu61kKcToLz0FapmHu Visit us, register and submit an abstract on our website! https://www.euroscipy.org/2016/ SciPythonic regards, The EuroSciPy 2016 team

1 1

ENH: compute many inner products quickly
by Scott Sievert June 6, 2016

June 6, 2016

I recently ran into an application where I had to compute many inner products quickly (roughy 50k inner products in less than a second). I wanted a vector of inner products over the 50k vectors, or `[x1.T @ A @ x1, …, xn.T @ A @ xn]` with A.shape = (1k, 1k). My first instinct was to look for a NumPy function to quickly compute this, such as np.inner. However, it looks like np.inner has some other behavior and I couldn’t get tensordot/einsum to work for me. Then a labmate pointed out that I can just do some slick matrix multiplication to compute the same quantity, `(X.T * A @ X.T).sum(axis=0)`. I opened [a PR] with this, and proposed that we define a new function called `inner_prods` for this. However, in the PR, @shoyer pointed out > The main challenge is to figure out how to transition the behavior of all these operations, while preserving backwards compatibility. Quite likely, we need to pick new names for these functions, though we should try to pick something that doesn't suggest that they are second class alternatives. Do we choose new function names? Do we add a keyword arg that changes what np.inner returns? [a PR]:https://github.com/numpy/numpy/pull/7690

8 9

Re: [Numpy-discussion] ENH: compute many inner products quickly
by Mark Daoust June 5, 2016

June 5, 2016

Here's the einsum version: `es = np.einsum('Na,ab,Nb->N',X,A,X)` But that's running ~45x slower than your version. OT: anyone know why einsum is so bad for this one? Mark Daoust On Sat, May 28, 2016 at 11:53 PM, Scott Sievert <sievert.scott(a)gmail.com> wrote: > I recently ran into an application where I had to compute many inner > products quickly (roughy 50k inner products in less than a second). I > wanted a vector of inner products over the 50k vectors, or `[x1.T @ A @ x1, > …, xn.T @ A @ xn]` with A.shape = (1k, 1k). > > My first instinct was to look for a NumPy function to quickly compute > this, such as np.inner. However, it looks like np.inner has some other > behavior and I couldn’t get tensordot/einsum to work for me. > > Then a labmate pointed out that I can just do some slick matrix > multiplication to compute the same quantity, `(X.T * A @ X.T).sum(axis=0)`. > I opened [a PR] with this, and proposed that we define a new function > called `inner_prods` for this. > > However, in the PR, @shoyer pointed out > > > The main challenge is to figure out how to transition the behavior of > all these operations, while preserving backwards compatibility. Quite > likely, we need to pick new names for these functions, though we should try > to pick something that doesn't suggest that they are second class > alternatives. > > Do we choose new function names? Do we add a keyword arg that changes what > np.inner returns? > > [a PR]:https://github.com/numpy/numpy/pull/7690 > > > > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion(a)scipy.org > https://mail.scipy.org/mailman/listinfo/numpy-discussion > >

3 2

PyArray_Scalar should not use memcpy
by Matti Picus June 4, 2016

June 4, 2016

Hi. This is a heads up and RFC about a pull request I am preparing for PyArray_Scalar, within the framework of getting NumPy working properly on PyPy. For those who don't know, the numpy HEAD builds and runs on PyPy2.7 HEAD (otherwise known as nightly default). However there are a number of test failures, some are caused by (ab)use of memcpy on c-level pointers obtained from Py*_FromString(). I am currently rewriting PyArray_Scalar to not use memcpy, and wondering how deep of a refactoring would be acceptable by the maintainers in a single pull request? Should I just stick to small changes to eliminate the two calls to memcpy, or clean up and restructure the entire function around a more switch(type_num) programming style? Thanks, Matti

2 1