Mailman 3 July 2013 - NumPy-Discussion

azip
by Alan G Isaac 18 Jul '13

18 Jul '13

I'm floating this thought even though it is not fleshed out. On occasion, I run into the following problem: I have a rectangular array A to which I want to append a (probably) one dimensional vector b to make [A|b]. Of course this can be done as np.hstack((x,b[:,None])) (or obscurely np.r_['1,2,0',x,b]), but this has the following issues: - what if ``b`` turns out to be a list? - what if ``b`` turns out to be 2d (e.g., a column vector)? - it's a bit ugly - it is not obvious when read by others (e.g., students) (The last is a key motivation for me to talk about this.) All of which leads me to wonder if there might be profit in a numpy.azip function that takes as arguments - a tuple of arraylike iterables - an axis along which to concatenate (say, like r_ does) iterated items To make that a little clearer (but not to provide a suggested implementation), it might behave something like def azip(alst, axis=1): results = [] for tpl in zip(*alst): results.append(np.r_[tpl]) return np.rollaxis(np.array(results), axis-1) Alan Isaac

5 9

Results of matplotlib user survey 2013
by Michael Droettboom 18 Jul '13

18 Jul '13

We have had 508 responses to the matplotlib user survey. Quite a nice turnout! You can view the results here: https://docs.google.com/spreadsheet/viewanalytics?key=0AjrPjlTMRTwTdHpQS25p… and from there, you can access the complete raw results. I will be doing more analysis of the results over the coming days and weeks, including dedup'ing some of the responses and converting some of the free-form responses into github issues etc. Volunteers to help with this are of course welcome! Cheers, Mike

1 1

ANN: HDF5 for Python (h5py) 2.2 BETA
by Andrew Collette 18 Jul '13

18 Jul '13

Announcing HDF5 for Python (h5py) 2.2.0 BETA ============================================ We are proud to announce that HDF5 for Python 2.2.0 (beta) is now available. Because of the large number of new features in this release, we are actively seeking community feedback over the (2-week) beta period. The h5py package is a Pythonic interface to the HDF5 binary data format. It lets you store huge amounts of numerical data, and easily manipulate that data from NumPy. For example, you can slice into multi-terabyte datasets stored on disk, as if they were real NumPy arrays. Thousands of datasets can be stored in a single file, categorized and tagged however you want. H5py uses straightforward NumPy and Python metaphors, like dictionary and NumPy array syntax. For example, you can iterate over datasets in a file, or check out the .shape or .dtype attributes of datasets. You don't need to know anything special about HDF5 to get started. Documentation and download links are available at: http://www.h5py.org Parallel HDF5 ============= This version of h5py introduces support for MPI/Parallel HDF5, using the mpi4py package. Parallel HDF5 is the native method for sharing files and objects across multiple processes. Unlike "multiprocessing" based solutions, all processes in an MPI-based program can read from and write to the same shared HDF5 file. There is a guide to using Parallel HDF5 at the h5py web site: http://h5py.org/docs/build/html/topics/mpi.html Other new features ================== * Support for Python 3.3 * Support for 16-bit "mini" floats * Access to the HDF5 scale-offset filter * Field names are now allowed when writing to a dataset * Region references now preserve the shape of their selections * File-resident "committed" types can be linked to datasets and attributes * A new "move" method on Group objects * Many new options for Group.copy

1 0

np.select use case
by Graeme B. Bell 18 Jul '13

18 Jul '13

Quick question: Can anyone think of a realistic/real-world use case for array broadcasting and np.select, (other than scalar to ndarray broadcasting)? e.g. differently shaped arrays with matching lower dimensions. (I don't know if a use case even exists). Graeme.

1 0

[ANN] 4th Python Symposium at AMS2014
by Jonathan Rocher 17 Jul '13

17 Jul '13

[Apologies for the cross-post] Dear all, If you work with Python around themes like big data, climate, meteorological or oceanic science, and/or GIS, you should come present at the 4th Python Symposium, as part of the American Meteorological Society conference in Atlanta in Feb 2014: http://annual.ametsoc.org/2014/index.cfm/programs-and-events/conferences-an… The *abstract deadline is Aug 1st*! Jonathan -- Jonathan Rocher, PhD Scientific software developer SciPy2013 conference co-chair Enthought, Inc. jrocher(a)enthought.com 1-512-536-1057 http://www.enthought.com

1 0

Size/Shape
by Brady McCary 17 Jul '13

17 Jul '13

NumPy Folks, Would someone please discuss or point me to a discussion about the discrepancy in size vs shape in the following MWE? In this example I have used a grayscale PNG version of the ImageMagick logo, but any image which is not square will do. $ python Python 2.7.4 (default, Apr 19 2013, 18:28:01) [GCC 4.7.3] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import Image >>> import matplotlib.pyplot as plt >>> >>> s = 'logo.png' >>> >>> im = Image.open(s) >>> ar = plt.imread(s) >>> >>> im.size (640, 480) >>> >>> ar.shape (480, 640) >>> The extents/shape of the NumPy array (as loaded by matplotlib, but this convention seems uniform through NumPy) are transposed from what seems to be the usual convention. Why was this choice made? Brady

2 1

Re: [Numpy-discussion] empty_like for masked arrays
by Gregorio Bastardo 17 Jul '13

17 Jul '13

Hi, On Mon, Jun 10, 2013 at 3:47 PM, Nathaniel Smith <njs(a)pobox.com> wrote: > Hi all, > > Is there anyone out there using numpy masked arrays, who has an > opinion on how empty_like (and its friends ones_like, zeros_like) > should handle the mask? > > Right now apparently if you call np.ma.empty_like on a masked array, > you get a new masked array that shares the original array's mask, so > modifying one modifies the other. That's almost certainly wrong. This > PR: > https://github.com/numpy/numpy/pull/3404 > makes it so instead the new array has values that are all set to > empty/zero/one, and a mask which is set to match the input array's > mask (so whenever something was masked in the original array, the > empty/zero/one in that place is also masked). We don't know if this is > the desired behaviour for these functions, though. Maybe it's more > intuitive for the new array to match the original array in shape and > dtype, but to always have an empty mask. Or maybe not. None of us > really use np.ma, so if you do and have an opinion then please speak > up... I recently joined the mailing list, so the message might not reach the original thread, sorry for that. I use masked arrays extensively, and would vote for the first option, as I use the *_like operations with the assumption that the resulting array has the same mask as the original. I think it's more intuitive than selecting between all masked or all unmasked behaviour. If it's not too late, please consider my use case. Thanks, Gregorio

2 1

ufunc overrides
by Blake Griffith 17 Jul '13

17 Jul '13

Hello NumPy, Part of my GSoC is compatibility with SciPy's sparse matrices and NumPy's ufuncs. Currently there is no feasible way to do this without changing ufuncs a bit. I've been considering a mechanism to override ufuncs based on checking the ufuncs arguments for a __ufunc_override__ attribute. Then handing off the operation to a function specified by that attribute. I prototyped this in python and did a demo in a blog post here: http://cwl.cx/posts/week-6-ufunc-overrides.html This is similar to a previously discussed, but never implemented change: http://mail.scipy.org/pipermail/numpy-discussion/2011-June/056945.html However it seems like the ufunc machinery might be ripped out and replaced with a true multi-method implementation soon. See Travis' blog post: http://technicaldiscovery.blogspot.com/2013/07/thoughts-after-scipy-2013-an… So I'd like to make my changes as forward compatible as possible. However I'm not sure what I should even consider here, or how forward compatible my current implementation is. Thoughts? Until then, I'm writing up a nep, it is still pretty incomplete, it can be found here: https://github.com/cowlicks/numpy/blob/ufunc-override/doc/neps/ufunc-overri…

3 2

restricting object arrays to a single Python type
by Geoffrey Irving 16 Jul '13

16 Jul '13

Is there a standard way of creating an object array restricted to a particular python type? I want a safe way of sending arrays of objects back and forth between Python and C++, and it'd be great if I could use numpy arrays on the Python side instead of creating a new type. For example, I might have a C++ class Force which is simultaneously a valid Python extension type (also named "Force"). I'd like to be able to switch between Array<Force> on the C++ side and a suitable numpy array on the Python side, while preventing Python from ever storing an object with different type (say, a tuple) in the array. Note that Array<Force> has the memory representation of an array of PyObject*'s. Thanks, Geoffrey

2 4

What should be the result in some statistics corner cases?
by Charles R Harris 15 Jul '13

15 Jul '13

Some corner cases in the mean, var, std. *Empty arrays* I think these cases should either raise an error or just return nan. Warnings seem ineffective to me as they are only issued once by default. In [3]: ones(0).mean() /home/charris/.local/lib/python2.7/site-packages/numpy/core/_methods.py:61: RuntimeWarning: invalid value encountered in double_scalars ret = ret / float(rcount) Out[3]: nan In [4]: ones(0).var() /home/charris/.local/lib/python2.7/site-packages/numpy/core/_methods.py:76: RuntimeWarning: invalid value encountered in true_divide out=arrmean, casting='unsafe', subok=False) /home/charris/.local/lib/python2.7/site-packages/numpy/core/_methods.py:100: RuntimeWarning: invalid value encountered in double_scalars ret = ret / float(rcount) Out[4]: nan In [5]: ones(0).std() /home/charris/.local/lib/python2.7/site-packages/numpy/core/_methods.py:76: RuntimeWarning: invalid value encountered in true_divide out=arrmean, casting='unsafe', subok=False) /home/charris/.local/lib/python2.7/site-packages/numpy/core/_methods.py:100: RuntimeWarning: invalid value encountered in double_scalars ret = ret / float(rcount) Out[5]: nan *ddof >= number of elements* I think these should just raise errors. The results for ddof >= #elements is happenstance, and certainly negative numbers should never be returned. In [6]: ones(2).var(ddof=2) /home/charris/.local/lib/python2.7/site-packages/numpy/core/_methods.py:100: RuntimeWarning: invalid value encountered in double_scalars ret = ret / float(rcount) Out[6]: nan In [7]: ones(2).var(ddof=3) Out[7]: -0.0 * nansum* Currently returns nan for empty arrays. I suspect it should return nan for slices that are all nan, but 0 for empty slices. That would make it consistent with sum in the empty case. Chuck

8 24