Mailman 3 February 2015 - NumPy-Discussion

So I found a bug...
by Sturla Molden Feb. 27, 2015

Feb. 27, 2015

Somewhere... But where is it? NumPy, SciPy, Matplotlib, Cython or ipython? I am suspecting ipython, but proving it is hard... http://nbviewer.ipython.org/urls/dl.dropboxusercontent.com/u/12464039/lenna… Sturla

3 6

ANN: SfePy 2015.1
by Robert Cimrman Feb. 26, 2015

Feb. 26, 2015

I am pleased to announce release 2015.1 of SfePy. Description ----------- SfePy (simple finite elements in Python) is a software for solving systems of coupled partial differential equations by the finite element method or by the isogeometric analysis (preliminary support). It is distributed under the new BSD license. Home page: http://sfepy.org Mailing list: http://groups.google.com/group/sfepy-devel Git (source) repository, issue tracker, wiki: http://github.com/sfepy Highlights of this release -------------------------- - support for multiple fields in isogeometric analysis - redesigned handling of solver parameters - new modal analysis example For full release notes see http://docs.sfepy.org/doc/release_notes.html#id1 (rather long and technical). Best regards, Robert Cimrman and Contributors (*) (*) Contributors to this release (alphabetical order): Lubos Kejzlar, Vladimir Lukes

1 0

GSoC'15 - mentors & ideas
by Ralf Gommers Feb. 26, 2015

Feb. 26, 2015

Hi all, On Fri, Feb 20, 2015 at 10:05 AM, Ralf Gommers <ralf.gommers(a)gmail.com> wrote: > Hi all, > > It's time to start preparing for this year's Google Summer of Code. There > is actually one urgent thing to be done (before 19.00 UTC today), which is > to get our ideas page in decent shape. It doesn't have to be final, but > there has to be enough on there for the organizers to judge it. This page > is here: https://github.com/scipy/scipy/wiki/GSoC-project-ideas. I'll be > reworking it and linking it from the PSF page today, but if you already > have new ideas please add them there. See > https://wiki.python.org/moin/SummerOfCode/OrgIdeasPageTemplate for this > year's template for adding a new idea. > The ideas page is now in pretty good shape. More ideas are very welcome though, especially easy or easy/intermediate ideas. Numpy right now has zero easy ones and Scipy only one and a half. What we also need is mentors. All ideas already have a potential mentor listed, however some ideas are from last year and I'm not sure that all those mentors really are available this year. And more than one potential mentor per idea is always good. So can everyone please add/remove his or her name on that page? I'm happy to take care of most of the organizational aspects this year, however I'll be offline for two weeks in July and from the end of August onwards, so I'll some help in those periods. Any volunteers? Thanks, Ralf

8 11

Objects exposing the array interface
by Jaime Fernández del Río Feb. 26, 2015

Feb. 26, 2015

An issue was raised yesterday in github, regarding np.may_share_memory when run on a class exposing an array using the __array__ method. You can check the details here: https://github.com/numpy/numpy/issues/5604 Looking into it, I found out that NumPy doesn't really treat objects exposing __array__,, __array_interface__, or __array_struct__ as if they were proper arrays: 1. When converting these objects to arrays using PyArray_Converter, if the arrays returned by any of the array interfaces is not C contiguous, aligned, and writeable, a copy that is will be made. Proper arrays and subclasses are passed unchanged. This is the source of the error reported above. 2. When converting these objects using PyArray_OutputConverter, as well as in similar code in the ufucn machinery, anything other than a proper array or subclass raises an error. This means that, contrary to what the docs on subclassing say, see below, you cannot use an object exposing the array interface as an output parameter to a ufunc The following classes can be used to test this behavior: class Foo: def __init__(self, arr): self.arr = arr def __array__(self): return self.arr class Bar: def __init__(self, arr): self.arr = arr self.__array_interface__ = arr.__array_interface__ class Baz: def __init__(self, arr): self.arr = arr self.__array_struct__ = arr.__array_struct__ They all behave the same with these examples: >>> a = Foo(np.ones(5)) >>> np.add(a, a) array([ 2., 2., 2., 2., 2.]) >>> np.add.accumulate(a) array([ 1., 2., 3., 4., 5.]) >>> np.add(a, a, out=a) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: return arrays must be of ArrayType >>> np.add.accumulate(a, out=a) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: output must be an array I think this should be changed, and whatever gets handed by this methods/interfaces be treated as if it were an array or subclass of it. This is actually what the docs on subclassing say about __array__ here: http://docs.scipy.org/doc/numpy/reference/arrays.classes.html#numpy.class._… This also seems to contradict a rather cryptic comment in the code of PyArray_GetArrayParamsFromObject, which is part of the call sequence of this whole mess, see here: https://github.com/numpy/numpy/blob/maintenance/1.9.x/numpy/core/src/multia… /* * If op supplies the __array__ function. * The documentation says this should produce a copy, so * we skip this method if writeable is true, because the intent * of writeable is to modify the operand. * XXX: If the implementation is wrong, and/or if actual * usage requires this behave differently, * this should be changed! */ There has already been some discussion in the issue linked above, but I would appreciate any other thoughts on the idea of treating objects with some form of array interface as if they were arrays. Does it need a deprecation cycle? Is there some case I am not considering where this could go horribly wrong? Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

2 3

PR, extended site.cfg capabilities
by Nick Papior Andersen Feb. 24, 2015

Feb. 24, 2015

Dear all, I have initiated a PR-5597 <https://github.com/numpy/numpy/pull/5597>, which enables the reading of new flags from the site.cfg file. @rgommers requested that I posted some information on this site, possibly somebody could test it on their setup. So the PR basically enables reading these extra options in each section: runtime_library_dirs : Add runtime library directories to the shared libraries (overrides the dreaded LD_LIBRARY_PATH) extra_compile_args: Adds extra compile flags to the compilation extra_link_args: Adds extra flags when linking to libraries Note that this PR will "fix" a lot of issues down the line. Specifically all software which utilises numpy's distutils will benefit from this. As an example, I have successfully set runtime_library_dirs for site.cfg in numpy, where scipy, petsc4py, pygsl, slepc4py utilises these flags and this enables me to create environments without the need for LD_LIBRARY_PATH. The other options simply adds to the flexibility of the compilation to test different optimisations etc. For instance my OpenBLAS section looks like this: [openblas] library_dirs = /opt/openblas/0.2.13/gnu-4.9.2/lib include_dirs = /opt/openblas/0.2.13/gnu-4.9.2/include runtime_library_dirs = /opt/openblas/0.2.13/gnu-4.9.2/lib I hope this can be of use to somebody else than me :) Feel free to test it and provide feedback! -- Kind regards Nick

2 4

np.nonzero behavior with multidimensional arrays
by Jaime Fernández del Río Feb. 23, 2015

Feb. 23, 2015

This was raised in SO today: http://stackoverflow.com/questions/28663142/why-is-np-wheres-result-read-on… np.nonzero (and np.where for boolean arrays) behave differently for 1-D and higher dimensional arrays: In the first case, a tuple with a single behaved base ndarray is returned: >>> a = np.ma.array(range(6)) >>> np.where(a > 3) (array([4, 5]),) >>> np.where(a > 3)[0].flags C_CONTIGUOUS : True F_CONTIGUOUS : True OWNDATA : True WRITEABLE : True ALIGNED : True UPDATEIFCOPY : False In the second, a tuple with as many arrays as dimensions in the passed array is returned, but the arrays are not base ndarrays, but of the same subtype as was passed to the function. These arrays are also set as non-writeable: >>> np.where(a.reshape(2, 3) > 3) (masked_array(data = [1 1], mask = False, fill_value = 999999) , masked_array(data = [1 2], mask = False, fill_value = 999999) ) >>> np.where(a.reshape(2, 3) > 3)[0].flags C_CONTIGUOUS : False F_CONTIGUOUS : False OWNDATA : False WRITEABLE : False ALIGNED : True UPDATEIFCOPY : False I can't think of any reason that justifies this difference, and believe they should be made to return similar results. My feeling is that the proper behavior is the 1-D one, and that the behavior for multidimensional arrays should match it. Anyone can think of any reason that justifies the current behavior? Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

3 3

One-byte string dtype: third time's the charm?
by Aldcroft, Thomas Feb. 23, 2015

Feb. 23, 2015

The idea of a one-byte string dtype has been extensively discussed twice before, with a lot of good input and ideas, but no action [1, 2]. tl;dr: Perfect is the enemy of good. Can numpy just add a one-byte string dtype named 's' that uses latin-1 encoding as a bridge to enable Python 3 usage in the near term? A key consequence of not having a one-byte string dtype is that handling ASCII data stored in binary formats such as HDF or FITS is basically broken in Python 3. Packages like h5py, pytables, and astropy.io.fits all return text data arrays with the numpy 'S' type, and in fact have no direct support for the numpy wide unicode 'U' type. In Python 3, the 'S' type array cannot be compared with the Python str type, so that something like below fails: >>> mask = (names_array == "john") # FAIL Problems like this are now showing up in the wild [3]. Workarounds are also showing up, like a way to easily convert from 'S' to 'U' within astropy Tables [4], but this is really not a desirable way to go. Gigabyte-sized string data arrays are not uncommon, so converting to UCS-4 is a real memory and performance hit. For a good top-level summary of much of the previous thread discussion, see [5] from Chris Barker. Condensing this down to just a few points: - *Changing* the behavior of the existing 'S' type is going to break code and seems a bad idea. - *Adding* a new dtype 's' will work and allow highly performant conversion from 'S' to 's' via view(). - Using the latin-1 encoding will minimize code breakage vis-a-vis what works in Python 2 [6]. Using latin-1 is a pragmatic compromise that provides continuity to allow scientists to run their existing code in Python 3 and have things just work. It isn't perfect and it should not be the end of the story, but it would be good. This single issue is the *only* thing blocking me and my team from using Python 3 in operations. As a final point, I don't know the numpy internals at all, but it *seems* like this proposal is one of the easiest to implement amongst those that were discussed. Cheers, Tom [1]: http://mail.scipy.org/pipermail/numpy-discussion/2014-January/068622.html [2]: http://mail.scipy.org/pipermail/numpy-discussion/2014-July/070574.html [3]: https://github.com/astropy/astropy/issues/3311 [4]: http://astropy.readthedocs.org/en/latest/api/astropy.table.Table.html#astro… [5]: http://mail.scipy.org/pipermail/numpy-discussion/2014-July/070631.html [6]: It is not uncommon to store uint8 data in a bytestring

6 16

Google Summer of Code and NumFOCUS
by Raniere Silva Feb. 20, 2015

Feb. 20, 2015

Hi, NumFOCUS has promotes and supports the ongoing research and development of open-source computing tools including NumPy. This year NumFOCUS want to try be a Google Summer of Code "umbrella" mentoring organization, Umbrella organizations are mentoring organizations accepted into the Google Summer of Code program that have other open source organizations working "under" them. Sometime organizations that work very closely or have very similar goals or communities may get put together under an "umbrella." Google stills expects all organizations under the umbrella, whether accepted into the program under their title or not, to adhere to all the rules and regulations of the program. From https://www.google-melange.com/gsoc/document/show/gsoc_program/google/gsoc2… To help promote and support NumPy. We encourage NumPy to apply to Google Summer of Code under your own title and will be very happy if you can also do with us. If you are interested, please check https://github.com/swcarpentry/gsoc2015 and https://github.com/swcarpentry/gsoc2015/blob/master/CONTRIBUTING.md. If you have any question, please email me directly. Thanks in advance, Raniere

2 2

GSoC'15 prep - ideas page
by Ralf Gommers Feb. 20, 2015

Feb. 20, 2015

Hi all, It's time to start preparing for this year's Google Summer of Code. There is actually one urgent thing to be done (before 19.00 UTC today), which is to get our ideas page in decent shape. It doesn't have to be final, but there has to be enough on there for the organizers to judge it. This page is here: https://github.com/scipy/scipy/wiki/GSoC-project-ideas. I'll be reworking it and linking it from the PSF page today, but if you already have new ideas please add them there. See https://wiki.python.org/moin/SummerOfCode/OrgIdeasPageTemplate for this year's template for adding a new idea. Cheers, Ralf

1 0

PyCon?
by Nathaniel Smith Feb. 18, 2015

Feb. 18, 2015

Hi all, It looks like I'll be at PyCon this year. Anyone else? Any interest in organizing a numpy sprint? -n -- Nathaniel J. Smith -- http://vorpus.org

2 1