NumPy-Discussion October 2013

numpy-discussion@python.org

78 participants
74 discussions

Re: [Numpy-discussion] [pydata] Re: [SciPy-Dev] 1.8.0rc1
by Nathaniel Smith 01 Oct '13

01 Oct '13

I bet the difference is: In master, nansum ultimately calls arr.sum(...), which will be intercepted by Series.sum. In 1.8.x, nansum ultimately calls np.add.reduce(...), which can't be intercepted and will return the wrong thing. AFAICT the np.add.reduce(a, ...) call could just be replaced with a.sum(...), but I might be missing something... surely there must have been some reason it was written that way in the first place? On Tue, Oct 1, 2013 at 2:48 PM, Jeff <jeffreback(a)gmail.com> wrote: > so...looks like a backport issue? > > > On Tuesday, October 1, 2013 6:41:26 AM UTC-4, Nathaniel Smith wrote: >> >> On Tue, Oct 1, 2013 at 3:27 AM, Charles R Harris >> <charles...(a)gmail.com> wrote: >> >>> >> On Mon, Sep 30, 2013 at 5:12 PM, Christoph Gohlke <cgo...(a)uci.edu >> >>> >> <mailto:cgo...@uci.edu>> wrote: >> >>> >> > NumPy 1.8.0rc1 looks good. All tests pass on Windows and >> >>> >> most >> >>> >> 3rd party >> >>> >> > packages test OK now. Thank you. >> >>> >> > >> >>> >> > A few tests still fail in the following packages when >> >>> >> run >> >>> >> with >> >>> >> > numpy-MKL-1.8.0rc1-win-amd64-py3.3 compared to >> >>> >> > numpy-MKL-1.7.1-win-amd64-py3.3: >> >>> >> > >> >>> >> > 1) Pandas 0.12.0 >> >>> >> > >> >>> >> > ``` >> >>> >> > >> >>> >> >> >>> >> >> >>> >> ====================================================================== >> >>> >> > FAIL: test_nansum_buglet >> >>> >> (pandas.tests.test_series.TestNanops) >> >>> >> > >> >>> >> >> >>> >> >> >>> >> ---------------------------------------------------------------------- >> >>> >> > Traceback (most recent call last): >> >>> >> > File >> >>> >> "X:\Python33\lib\site-packages\pandas\tests\test_series.py", >> >>> >> > line 254, in test_nansum_buglet >> >>> >> > assert_almost_equal(result, 1) >> >>> >> > File >> >>> >> "X:\Python33\lib\site-packages\pandas\util\testing.py", line >> >>> >> > 134, in assert_almost_equal >> >>> >> > np.testing.assert_(isiterable(b)) >> >>> >> > File >> >>> >> "D:\Dev\Compile\Test\numpy-build\numpy\testing\utils.py", line >> >>> >> > 44, in assert_ >> >>> >> > raise AssertionError(msg) >> >>> >> > AssertionError >> >>> >> > ``` >> >>> >> > >> >>> >> > Possibly related: >> >>> >> > >> >>> >> > ``` >> >>> >> > >>> import numpy as np >> >>> >> > >>> from pandas import Series >> >>> >> > >>> s = Series([0.0]) >> >>> >> > >>> result = np.nansum(s) >> >>> >> > >>> print(result) >> >>> >> > Traceback (most recent call last): >> >>> >> > File "<stdin>", line 1, in <module> >> >>> >> > File >> >>> >> "X:\Python33\lib\site-packages\pandas\core\base.py", line >> >>> >> > 19, in >> >>> >> > __str__ >> >>> >> > return self.__unicode__() >> >>> >> > File >> >>> >> "X:\Python33\lib\site-packages\pandas\core\series.py", line >> >>> >> > 1115, in __unicode__ >> >>> >> > length=len(self) > 50, >> >>> >> > TypeError: len() of unsized object >> >>> >> > ``` >> [...] >> > >> > The pandas test passes for current pandas dev, so it looks like a bug on >> > their end that has been taken care of. >> > >> > test_nansum_buglet (__main__.TestNanops) ... ok >> >> I'm concerned about this. 0.12.0 is currently the latest pandas >> release, so even if it is a bug on their side, we're going to be >> converting it from a latent bug to a real bug when we release... >> CC'ing pydata, do you guys have any insight into what changed here? >> >> The code is: >> s = pandas.Series([1.0, np.nan]) >> result = np.nansum(s) >> With numpy 1.7.1, 'result' comes out as a np.float64. With numpy >> maintenance/1.8.x, 'result' comes out as a 0-d Series object. Series >> is a subclass of ndarray, but it's supposed to always be 1-d, so all >> kinds of stuff blows up as soon as you have a 0-d Series object. >> >> I'm not sure what changed in numpy's nansum; if I try this same test >> with a simple no-op ndarray subclass: >> class MyArray(np.ndarray): >> pass >> np.nansum(np.array([1.0, np.nan]).view(MyArray)) >> then 1.7.1 and maintenance/1.8.x both act the same, and both return a >> 0-d MyArray, so it's not just a question of whether we remembered to >> handle subclasses at all. >> >> -n > > -- > You received this message because you are subscribed to the Google Groups > "PyData" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to pydata+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out.

2 2

Re: [Numpy-discussion] [pydata] Re: [SciPy-Dev] 1.8.0rc1
by Nathaniel Smith 01 Oct '13

01 Oct '13

On Tue, Oct 1, 2013 at 2:39 PM, Jeff <jeffreback(a)gmail.com> wrote: > Just built from numpy master looks ok > > In [7]: pd.__version__ > Out[7]: '0.12.0' > > In [8]: np.__version__ > Out[8]: '1.9.0.dev-74abfa2' > > In [9]: s = pandas.Series([1.0, np.nan]) > > In [10]: np.nansum(s) > Out[10]: 1.0 > > In [11]: type(np.nansum(s)) > Out[11]: numpy.float64 Yeah, I can confirm this... With pandas 0.12.0, this code: np.nansum(pandas.Series([1.0, np.nan])) works correctly using current master (74abfa27), but is broken with current maintenance/1.8.x (f5fe91b9). -n > On Tuesday, October 1, 2013 6:41:26 AM UTC-4, Nathaniel Smith wrote: >> >> On Tue, Oct 1, 2013 at 3:27 AM, Charles R Harris >> <charles...(a)gmail.com> wrote: >> >>> >> On Mon, Sep 30, 2013 at 5:12 PM, Christoph Gohlke <cgo...(a)uci.edu >> >>> >> <mailto:cgo...@uci.edu>> wrote: >> >>> >> > NumPy 1.8.0rc1 looks good. All tests pass on Windows and >> >>> >> most >> >>> >> 3rd party >> >>> >> > packages test OK now. Thank you. >> >>> >> > >> >>> >> > A few tests still fail in the following packages when >> >>> >> run >> >>> >> with >> >>> >> > numpy-MKL-1.8.0rc1-win-amd64-py3.3 compared to >> >>> >> > numpy-MKL-1.7.1-win-amd64-py3.3: >> >>> >> > >> >>> >> > 1) Pandas 0.12.0 >> >>> >> > >> >>> >> > ``` >> >>> >> > >> >>> >> >> >>> >> >> >>> >> ====================================================================== >> >>> >> > FAIL: test_nansum_buglet >> >>> >> (pandas.tests.test_series.TestNanops) >> >>> >> > >> >>> >> >> >>> >> >> >>> >> ---------------------------------------------------------------------- >> >>> >> > Traceback (most recent call last): >> >>> >> > File >> >>> >> "X:\Python33\lib\site-packages\pandas\tests\test_series.py", >> >>> >> > line 254, in test_nansum_buglet >> >>> >> > assert_almost_equal(result, 1) >> >>> >> > File >> >>> >> "X:\Python33\lib\site-packages\pandas\util\testing.py", line >> >>> >> > 134, in assert_almost_equal >> >>> >> > np.testing.assert_(isiterable(b)) >> >>> >> > File >> >>> >> "D:\Dev\Compile\Test\numpy-build\numpy\testing\utils.py", line >> >>> >> > 44, in assert_ >> >>> >> > raise AssertionError(msg) >> >>> >> > AssertionError >> >>> >> > ``` >> >>> >> > >> >>> >> > Possibly related: >> >>> >> > >> >>> >> > ``` >> >>> >> > >>> import numpy as np >> >>> >> > >>> from pandas import Series >> >>> >> > >>> s = Series([0.0]) >> >>> >> > >>> result = np.nansum(s) >> >>> >> > >>> print(result) >> >>> >> > Traceback (most recent call last): >> >>> >> > File "<stdin>", line 1, in <module> >> >>> >> > File >> >>> >> "X:\Python33\lib\site-packages\pandas\core\base.py", line >> >>> >> > 19, in >> >>> >> > __str__ >> >>> >> > return self.__unicode__() >> >>> >> > File >> >>> >> "X:\Python33\lib\site-packages\pandas\core\series.py", line >> >>> >> > 1115, in __unicode__ >> >>> >> > length=len(self) > 50, >> >>> >> > TypeError: len() of unsized object >> >>> >> > ``` >> [...] >> > >> > The pandas test passes for current pandas dev, so it looks like a bug on >> > their end that has been taken care of. >> > >> > test_nansum_buglet (__main__.TestNanops) ... ok >> >> I'm concerned about this. 0.12.0 is currently the latest pandas >> release, so even if it is a bug on their side, we're going to be >> converting it from a latent bug to a real bug when we release... >> CC'ing pydata, do you guys have any insight into what changed here? >> >> The code is: >> s = pandas.Series([1.0, np.nan]) >> result = np.nansum(s) >> With numpy 1.7.1, 'result' comes out as a np.float64. With numpy >> maintenance/1.8.x, 'result' comes out as a 0-d Series object. Series >> is a subclass of ndarray, but it's supposed to always be 1-d, so all >> kinds of stuff blows up as soon as you have a 0-d Series object. >> >> I'm not sure what changed in numpy's nansum; if I try this same test >> with a simple no-op ndarray subclass: >> class MyArray(np.ndarray): >> pass >> np.nansum(np.array([1.0, np.nan]).view(MyArray)) >> then 1.7.1 and maintenance/1.8.x both act the same, and both return a >> 0-d MyArray, so it's not just a question of whether we remembered to >> handle subclasses at all. >> >> -n > > -- > You received this message because you are subscribed to the Google Groups > "PyData" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to pydata+unsubscribe(a)googlegroups.com. > For more options, visit https://groups.google.com/groups/opt_out.

1 0

review/approval request
by Graeme B. Bell 01 Oct '13

01 Oct '13

Regarding 1.8: A few months ago I submitted a patch that fixes some bugs, including a major bug in select() that has been around for years, and optimised the code quite a bit. https://github.com/numpy/numpy/pull/3537 It would be nice to see the patch in 1.8, but it has languished for a couple of months waiting for final approval/merging. The patch is pretty small and the new code is not complicated. Are there any core devs that would have time/interest to help me get this approved/merged into numpy master? I have already adjusted the patch as suggested by njsmith to suit inclusion in master. Graeme. >>> 331701 >> >> I can't tell if these are real bugs in numpy, or tests checking that >> bottleneck is bug-for-bug compatible with old numpy and we just fixed >> some bugs, or what. It's clearly something to do with the >> nanarg{max,min} rewrite -- @charris, do you know what's going on here? >> >> > Yes ;) The previous behaviour of nanarg for all-nan axis was to cast nan to > intp when the result was an array, and return nan when a scalar. The > current behaviour is to return the most negative value of intp as an error > marker in both cases and raise a warning. It is a change in behavior, but I > think one that needs to be made. > > Chuck >

1 0

Issue with np.median and array subclasses in 1.8.0rc (worked with 1.7.0)
by Thomas Robitaille 01 Oct '13

01 Oct '13

Hi, The behavior for ``np.median`` and array sub-classes has changed in 1.8.0rc, which breaks unit-handling code (such as the ``quantities`` package, or ``astropy.units``): https://github.com/numpy/numpy/issues/3846 This previously worked from Numpy 1.5 (at least) to Numpy 1.7. Is there a new (and better) way to override the ``np.median`` behavior? Cheers, Tom

1 0

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

NumPy-Discussion October 2013