Mailman 3 October 2014 - NumPy-Discussion

ANN: Bokeh 0.6.1 release
by Damian Avila 02 Dec '14

02 Dec '14

On behalf of the Bokeh team, I am very happy to announce the release of Bokeh version 0.6.1! Bokeh is a Python library for visualizing large and realtime datasets on the web. Its goal is to provide to developers (and domain experts) with capabilities to easily create novel and powerful visualizations that extract insight from local or remote (possibly large) data sets, and to easily publish those visualization to the web for others to explore and interact with. This point release includes several bug fixes and improvements over our most recent 0.6.0 release: * Toolbar enhancements * bokeh-server fixes * Improved documentation * Button widgets * Google map support in the Python side * Code cleanup in the JS side and examples * New examples See the CHANGELOG for full details. In upcoming releases, you should expect to see more new layout capabilities (colorbar axes, better grid plots and improved annotations), additional tools, even more widgets and more charts, R language bindings, Blaze integration and cloud hosting for Bokeh apps. Don't forget to check out the full documentation, interactive gallery, and tutorial at http://bokeh.pydata.org as well as the Bokeh IPython notebook nbviewer index (including all the tutorials) at: http://nbviewer.ipython.org/github/ContinuumIO/bokeh-notebooks/blob/master/… If you are using Anaconda or miniconda, you can install with conda: conda install bokeh Alternatively, you can install with pip: pip install bokeh BokehJS is also available by CDN for use in standalone javascript applications: http://cdn.pydata.org/bokeh-0.6.1.min.js http://cdn.pydata.org/bokeh-0.6.1.min.css Issues, enhancement requests, and pull requests can be made on the Bokeh Github page: https://github.com/continuumio/bokeh Questions can be directed to the Bokeh mailing list: bokeh(a)continuum.io If you have interest in helping to develop Bokeh, please get involved! Cheers, Damián Avila damian.avila(a)continuum.io

6 7

Add `nrows` to `genfromtxt`
by Jaime Fernández del Río 04 Nov '14

04 Nov '14

There is a PR in github that adds a new keyword to the genfromtxt function, to limit the number of rows that actually get read in: https://github.com/numpy/numpy/pull/5103 It is mostly ready to go, and several devs have looked at it without complaining. Since it is an API change, I wanted to check here: if no one has any strong opposition, I will be merging it sometime tomorrow. You have been warned... ;-) Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

5 16

Choosing between NumPy and SciPy functions
by dmmcf＠dmmcf.net 31 Oct '14

31 Oct '14

A recent post raised a question about differences in results obtained with numpy.linalg.eigh() and scipy.linalg.eigh(), documented at http://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.eigh.html#… and http://docs.scipy.org/doc/scipy/reference/generated/scipy.linalg.eigh.html#…, respectively. It is clear that these functions address different mathematical problems (among other things, the SciPy routine can solve the generalized as well as standard eigenproblems); I am not concerned here with numerical differences in the results for problems both should be able to solve (the author of the original post received useful replies in that thread). What I would like to ask about is the situation this illustrates, where both NumPy and SciPy provide similar functionality (sometimes identical, to judge by the documentation). Is there some guidance on which is to be preferred? I could argue that using only NumPy when possible avoids unnecessary dependence on SciPy in some code, or that using SciPy consistently makes for a single interface and so is less error prone. Is there a rule of thumb for cases where SciPy names shadow NumPy names? I've used Python for a long time, but have only recently returned to doing serious numerical work with it. The tools are very much improved, but sometimes, like now, I feel I'm missing the obvious. I would appreciate pointers to any relevant documentation, or just a summary of conventional wisdom on the topic. Regards, Michael

10 17

[ANN - JOB] Assistant Researcher - Berkeley Institute for Data Science
by Fernando Perez 30 Oct '14

30 Oct '14

Hi all, the newly founded Berkeley Institute for Data Science is hiring researchers with a focus on open source tools for scientific computing, please see here for details: https://aprecruit.berkeley.edu/apply/JPF00590 Cheers, f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail

1 0

help using np.einsum for stacked matrix multiplication
by Andrew Nelson 30 Oct '14

30 Oct '14

Dear list, I have a 4D array, A, that has the shape (NX, NY, 2, 2). I wish to perform matrix multiplication of the 'NY' 2x2 matrices, resulting in the matrix B. B would have shape (NX, 2, 2). I believe that np.einsum would be up to the task, but I'm not quite sure of the subscripts I would need to achieve this. Can anyone help, please? cheers, Andrew. -- _____________________________________ Dr. Andrew Nelson _____________________________________

5 5

Re: [Numpy-discussion] help using np.einsum for stacked matrix multiplication
by Andrew Nelson 30 Oct '14

30 Oct '14

On Wed, Oct 29, 2014 at 10:39 AM, Andrew Nelson <andyfaff(a)gmail.com> wrote: > Dear list, > I have a 4D array, A, that has the shape (NX, NY, 2, 2). I wish to > perform matrix multiplication of the 'NY' 2x2 matrices, resulting in the > matrix B. B would have shape (NX, 2, 2). I believe that np.einsum would > be up to the task, but I'm not quite sure of the subscripts I would need to > achieve this. Ok, I'll try to explain in more detail of what I'm trying to do (I'm not skilled in matrix algebra). Say I have a series of matrices, M, which are all 2x2: M_0, M_1, ..., M_{NY - 1}. These all need to be multiplied by each other. i.e. N = M_0 x M_1 x ... x M_{NY - 1}. Note that I want to multiply M_0 by M_1, the result of that by M_2, the result of that by M_3 and so on. I can hold the (NY) matrices in a single array that has shape (NY, 2, 2). The first row in that array would be M_0, the last would be M_{NY-1}. The output of all that matrix multiplication would be a single 2x2 matrix. So I would've thought an operation would do something like this: #there are NY-1 matrix multiplications involved here. M[NY, 2, 2] -----> N[2, 2] Now let's make the next level of complication, I have NX of those M[NY, 2, 2] matrices. So I need to do the above matrix multiplication series NX times. I could hold all this in an array, P, with shape (NX, NY, 2, 2). Each of the NX rows are independent. Currently I am doing this in a nested loop, pseudocode follows: <python> output = np.zeros((NX, 2, 2)) for i in range(NX): temp = np.identity(2) for j in range(NY): temp = np.dot(temp, P[i, j]) output[i] = temp </python> My original question was posted as I would like to remove that doubly nested loop, with something more elegant, as well as a whole load faster. Is there an np.einsum that can furnish that? (Please forgive me if this still isn't clear or precise enough). -- _____________________________________ Dr. Andrew Nelson _____________________________________

1 0

Deprecate pkgload, PackageLoader
by Charles R Harris 30 Oct '14

30 Oct '14

Hi All, It is proposed to deprecate, then remove, pkgload and PackageLoader. Complaints? Cries of Anguish? Chuck

4 4

Re: [Numpy-discussion] Why ndarray provides four ways to flatten?
by Alexander Belopolsky 30 Oct '14

30 Oct '14

On Mon, Oct 27, 2014 at 9:41 PM, Yuxiang Wang <yw5aj(a)virginia.edu> wrote: > In my opinion - because they don't do the same thing, especially when > you think in terms in lower-level. > > ndarray.flat returns an iterator; ndarray.flatten() returns a copy; > ndarray.ravel() only makes copies when necessary; ndarray.reshape() is > more general purpose, even though you can use it to flatten arrays. > Out of the four ways, I find x.flat the most confusing. Unfortunately, it is also the most obvious name for the operation (and "ravel" is the least, but it is the fault of the English language where "to ravel" means "to unravel."). What x.flat returns, is not really an iterator. It is some hybrid between a view and an iterator. Consider this: >>> x = numpy.arange(6).reshape((2,3)) >>> i = x.flat >>> i.next() 0 >>> i.next() 1 >>> i.next() 2 So far no surprises, but what should i[0] return now? If you think of i as a C pointer you would expect 3, but >>> i[0] 0 What is worse, the above resets the index and now >>> i.index 0 OK, so now I expect that i[5] will reset the index to 5, but no >>> i[5] 5 >>> i.index 0 When would you prefer to use x.flat over x.ravel()? Is x.reshape(-1) always equivalent to x.ravel()? What is x.flat.copy()? Is it the same as x.flatten()? Why does flatiter even have a .copy() method? Isn't i.copy() the same as i.base.flatten(), only slower? And with all these methods, I still don't have the one that would flatten any array including a nested array like this: >>> x = np.array([np.arange(2), np.arange(3), np.arange(4)]) I need yet another function here, for example >>> np.hstack(x) array([0, 1, 0, 1, 2, 0, 1, 2, 3]) and what if I want to flatten a higher dimensional nested array, say >>> y = np.array([x[:1],x[:2],x]) can I do better than >>> np.hstack(np.hstack(y)) array([0, 1, 0, 1, 0, 1, 2, 0, 1, 0, 1, 2, 0, 1, 2, 3]) ?

6 14

Memory efficient alternative for np.loadtxt and np.genfromtxt
by Saullo Castro 29 Oct '14

29 Oct '14

I would like to start working on a memory efficient alternative for np.loadtxt and np.genfromtxt that uses arrays instead of lists to store the data while the file iterator is exhausted. The motivation came from this SO question: http://stackoverflow.com/q/26569852/832621 where for huge arrays the current NumPy ASCII readers are really slow and require ~6 times more memory. This case I tested with Pandas' read_csv() and it required 2 times more memory. I would be glad if you could share your experience on this matter. Greetings, Saullo

11 13

[ANN] Python for Scientific Computing conference in Boulder, CO; April'15
by Fernando Perez 29 Oct '14

29 Oct '14

Hi folks, a colleague from NCAR in Boulder just sent me this link about a conference they are organizing in the spring: https://sea.ucar.edu/conference/2015 I figured this might be of interest to many on these lists. The actual call isn't up yet, so if you're interested, watch that site for an upcoming call when they post it (I'm not directly involved, just passing the message along). Cheers f -- Fernando Perez (@fperez_org; http://fperez.org) fperez.net-at-gmail: mailing lists only (I ignore this when swamped!) fernando.perez-at-berkeley: contact me here for any direct mail

2 2