Mailman 3 January 2015 - NumPy-Discussion

Views of a different dtype
by Jaime Fernández del Río 04 Feb '15

04 Feb '15

HI all, There has been some recent discussion going on on the limitations that numpy imposes to taking views of an array with a different dtype. As of right now, you can basically only take a view of an array if it has no Python objects and neither the old nor the new dtype are structured. Furthermore, the array has to be either C or Fortran contiguous. This seem to be way too strict, but the potential for disaster getting a loosening of the restrictions wrong is big, so it should be handled with care. Allan Haldane and myself have been looking into this separately and discussing some of the details over at github, and we both think that the only true limitation that has to be imposed is that the offsets of Python objects within the new and old dtypes remain compatible. I have expanded Allan's work from here: https://github.com/ahaldane/numpy/commit/e9ca367 to make it as flexible as I have been able. An implementation of the algorithm in Python, with a few tests, can be found here: https://gist.github.com/jaimefrio/b4dae59fa09fccd9638c#file-dtype_compat-py I would appreciate getting some eyes on it for correctness, and to make sure that it won't break with some weird dtype. I am also trying to figure out what the ground rules for stride and shape conversions when taking a view with a different dtype should be. I submitted a PR (gh-5508) a couple for days ago working on that, but I am not so sure that the logic is completely sound. Again, to get more eyes on it, I am going to reproduce my thoughts here on the hope of getting some feedback. The objective would be to always allow a view of a different dtype (given that the dtypes be compatible as described above) to be taken if: - The itemsize of the dtype doesn't change. - The itemsize changes, but the array being viewed is the result of slicing and transposing axes of a contiguous array, and it is still contiguous, defined as stride == dtype.itemsize, along its smallest-strided dimension, and the itemsize of the newtype exactly divides the size of that dimension. - Ideally taking a view should be a reversible process, i.e. if oldtype = arr.dtype, then doing arr.view(newtype).view(oldtype) should give you back a view of arr with the same original shape, strides and dtype. This last point can get tricky if the minimal stride dimension has size 1, as there could be several of those, e.g.: >>> a = np.ones((3, 4, 1), dtype=float)[:, :2, :].transpose(0, 2, 1) >>> a.flags.contiguous False >>> a.shape (3, 1, 2) >>> a.strides # the stride of the size 1 dimension could be anything, ignore it! (32, 8, 8) b = a.view(complex) # this fails right now, but should work >>> b.flags.contiguous False >>> b.shape (3, 1, 1) >>> b.strides # the stride of the size 1 dimensions could be anything, ignore them! (32, 16, 16) c = b.view(float) # which of the two size 1 dimensions should we expand? "In the face of ambiguity refuse the temptation to guess" dictates that last view should raise an error, unless we agree and document some default. Any thoughts? Then there is the endless complication one could get into with arrays created with as_strided. I'm not smart enough to figure when and when not those could work, but am willing to retake the discussion if someone wiser si interested. With all these in mind, my proposal for the new behavior is that taking a view of an array with a different dtype would require: 1. That the newtype and oldtype be compatible, as defined by the algorithm checking object offsets linked above. 2. If newtype.itemsize == oldtype.itemsize no more checks are needed, make it happen! 3. If the array is C/Fortran contiguous, check that the size in bytes of the last/first dimension is evenly divided by newtype.itemsize. If it does, go for it. 4. For non-contiguous arrays: 1. Ignoring dimensions of size 1, check that no stride is smaller than either oldtype.itemsize or newtype.itemsize. If any is found this is an as_strided product, sorry, can't do it! 2. Ignoring dimensions of size 1, find a contiguous dimension, i.e. stride == oldtype.itemsize 1. If found, check that it is the only one with that stride, that it is the minimal stride, and that the size in bytes of that dimension is evenly divided by newitem,itemsize. 2. If none is found, check if there is a size 1 dimension that is also unique (unless we agree on a default, as mentioned above) and that newtype.itemsize evenly divides oldtype.itemsize. Apologies for the long, dense content, but any thought or comments are very welcome. Jaime -- (\__/) ( O.o) ( > <) Este es Conejo. Copia a Conejo en tu firma y ayúdale en sus planes de dominación mundial.

6 13

missing FloatingPointError for numpy on cygwin64
by Sebastien Gouezel 02 Feb '15

02 Feb '15

Dear all, I tried to use numpy (version 1.9.1, installed by `pip install numpy`) on cygwin64. I encountered the following weird bug: >>> import numpy >>> with numpy.errstate(all='raise'): ... print 1/float64(0.0) inf I was expecting a FloatingPointError, but it didn't show up. Curiously, with different numerical types (all intxx, or float128), I indeed get the FloatingPointError. Same thing with the most recent git version, or with 1.7.1 provided as a precompiled package by cygwin. This behavior does not happen on cygwin32 (I always get the FloatingPointError there). I wonder if there is something weird with my config, or if this is a genuine reproducible bug. If so, where should I start looking if I want to fix it? (I don't know anything about numpy's code) Sebastien

2 3

Numpy 'None' comparison FutureWarning
by Demitri Muna 31 Jan '15

31 Jan '15

Hi, I just encountered the following in my code: FutureWarning: comparison to `None` will result in an elementwise object comparison in the future. I'm very concerned about this. This is a very common programming pattern (lazy loading): class A(object): def __init__(self): self._some_array = None @property def some_array(self): if self._some_array == None: # perform some expensive setup of array return self._some_array It seems to me that the new behavior will break this pattern. I think that redefining the "==" operator is a little too aggressive here. It strikes me as very nonstandard and not at all obvious to someone reading the code that the comparison is a very special case for numpy objects. Unless there's some aspect I'm missing here, I think an element-wise comparator should be more explicit. Cheers, Demitri _________________________________________ Demitri Muna Department of Astronomy Le Ohio State University http://trillianverse.org http://scicoder.org

5 6

question np.partition
by Neal Becker 30 Jan '15

30 Jan '15

It sounds like np.partition could be used to answer the question: give me the highest K elements in a vector. Is this a correct interpretation? Something like partial sort, but returned elements are unsorted. I could really make some use of this, but in my case it is a list of objects I need to sort on a particular key. Is this algorithm available in general python code (not specific to numpy arrays)? -- -- Those who don't understand recursion are doomed to repeat it

3 2

Re: [Numpy-discussion] Testing of scipy
by Carl Kleffner 30 Jan '15

30 Jan '15

Hi Colin. this is an interesting test with different hardware. As a summary: - Python-2.7 amd64 - numpy-0.1.9.openblas: OK - scipy-0.15.1.openblas: 2 errors 11 failures - CPU: AMD A8-5600K APU (piledriver) scipy errors and failures due to piledriver: (1) ERROR: test_improvement (test_quadpack.TestCtypesQuad) WindowsError: [Error 193] %1 is not a valid Win32 application (2) ERROR: test_typical (test_quadpack.TestCtypesQuad) WindowsError: [Error 193] %1 is not a valid Win32 application (3) FAIL: test_interpolate.TestInterp1D.test_circular_refs ReferenceError: Remaining reference(s) to object (4) FAIL: test__gcutils.test_assert_deallocated ReferenceError: Remaining reference(s) to object other failures are known failures due to mingw-w64 / openblas build. (1) and (2) is a problem with ctypes.util.find_msvcrt(). This methods seems to be buggy. Not a scipy problem. Maybe the test could be enhanced. (3) and (4) is a problem due to garbage collection. No idea why. Maybe you can file a bug for (1) ... (4) Carl 2015-01-29 19:36 GMT+01:00 cjw <cjw(a)ncf.ca>: > Carl, > > I have already sent the test result for numpy. Here is the test result > for scipy. > > I hope that it helps. > > Colin W. > > ------------------------------ > *** Python 2.7.9 (default, Dec 10 2014, 12:28:03) [MSC v.1500 64 bit > (AMD64)] on win32. *** > >>> > [Dbg]>>> > C:\Python27\lib\site-packages\numpy\core\__init__.py:6: Warning: Numpy > 64bit experimental build with Mingw-w64 and OpenBlas. > from . import multiarray > Running unit tests for scipy > NumPy version 1.9.1 > NumPy is installed in C:\Python27\lib\site-packages\numpy > SciPy version 0.15.1 > SciPy is installed in C:\Python27\lib\site-packages\scipy > Python version 2.7.9 (default, Dec 10 2014, 12:28:03) [MSC v.1500 64 bit > (AMD64)] > nose version 1.3.4 > C:\Python27\lib\site-packages\numpy\lib\utils.py:95: DeprecationWarning: > `scipy.lib.blas` is deprecated, use `scipy.linalg.blas` instead! > warnings.warn(depdoc, DeprecationWarning) > C:\Python27\lib\site-packages\numpy\lib\utils.py:95: DeprecationWarning: > `scipy.lib.lapack` is deprecated, use `scipy.linalg.lapack` instead! > warnings.warn(depdoc, DeprecationWarning) > C:\Python27\lib\site-packages\numpy\lib\utils.py:95: DeprecationWarning: > `scipy.weave` is deprecated, use `weave` instead! > warnings.warn(depdoc, DeprecationWarning) > ...............................................................................................................................................................................................................................................................K......................................................................................................................................................................................................................................................................................EE.........................................................................................................................................F.....K..K............................................................................................................................................................................................................................................................................................................... > ....... > ....................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................SSSSSS......SSSSSS......SSSS......F....................... > ........ > ............................................................................F............S..........K..........................................................................................................................................................................................................................................F....................................................K.............................................................................................................................................................................................K...............................................................................................SSSSSSSS.K.................S................................................................................................................................................................................................................................................................................................ > ........ > .....................................S........................................................................................................................................................................................C:\Python27\lib\site-packages\scipy\optimize\minpack.py:604: > OptimizeWarning: Covariance of the parameters could not be estimated > category=OptimizeWarning) > .....................................................................................................................................................F.................K.F.................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................... > ....... > ......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................F.....................................................................................................................................................................................................................................................SS..........S....S...SSSSS.SSSSSSSSSS.....SSS..S.....S....K...........S..S........SSSSK...SSSS..S.........S........SS..........S...S...SSSSS.SSSSSSSSSS.....SSS........S................S..S...... > ..SSSS.. > .SSSS..S............S..............................S.................................................S........................S..........S................................S...................................................S........................S..........S..........SS.......S...S...SSSSS.SSSSSSSSSS.....SSS..S.....S............SS..S..S........SSSS....SSSS..S.........S......K.....................S.....K...................S..................SS...S.........................S..........S.........................S.................................................SS..S...........................S.........S........SS..........S....S...SSSSS.SSSSSSSSSS.....SSS..S.....S....K...........S..S........SSSSK...SSSS..S.........S........SS..........S...S...SSSSS.SSSSSSSSSS.....SSS........S................S..S........SSSS...SSSS..S............S..............................S.................................................S........................S..........S................................S... > ........ > ........................................S........................S..........S..........SS.......S...S...SSSSS.SSSSSSSSSS.....SSS..S.....S............SS..S..S........SSSS....SSSS..S.........S......K.....................S.....K...................S..................SS...S.........................S..........S.........................S.................................................SS..S...........................S.........S........SS..........S....S...SSSSS.SSSSSSSSSS.....SSS..S.....S....K...........S..S........SSSSK...SSSS..S.........S........SS..........S...S...SSSSS.SSSSSSSSSS.....SSS........S................S..S........SSSS...SSSS..S............S..............................S.................................................S........................S..........S................................S...................................................S........................S..........S..........SS.......S...S...SSSSS.SSSSSSSSSS.....SSS..S.....S............SS..S..S........SSSS....S > SSS..S.. > .......S......K.....................S.....K...................S..................SS...S.........................S..........S.........................S.................................................SS..S...........................S.........S........SS..........S....S...SSSSS.SSSSSSSSSS.....SSS..S.....S....K...........S..S........SSSSK...SSSS..S.........S........SS..........S...S...SSSSS.SSSSSSSSSS.....SSS........S................S..S........SSSS...SSSS..S............S..............................S.................................................S........................S..........S................................S...................................................S........................S..........S..........SS.......S...S...SSSSS.SSSSSSSSSS.....SSS..S.....S............SS..S..S........SSSS....SSSS..S.........S......K.....................S.....K...................S..................SS...S.........................S..........S.........................S........................ > ........ > .................SS..S...........................S.........S........SS..........S...S...SSSSS.SSSSSSSSSS.....SSS..S.....S....K...........S..S........SSSSK...SSSS..S.........S........SS..........S...S...SSSSS.SSSSSSSSSS.....SSS........S................S..S........SSSS...SSSS..S............S..............................S.................................................S........................S..........S................................S...................................................S........................S..........S..........SS.......S...S...SSSSS.SSSSSSSSSS.....SSS..S.....S............SS..S..S........SSSS....SSSS..S.........S......K.....................S.....K...................S..................SS...S.........................S..........S.........................S.................................................SS..S...........................S.........S..............SSSSSSSSSSSSSSSSSSSSSSSSSSSSS...........S....S......SSSS.SSSSSSSSSS.......SSS..S..........S....K..... > ........ > .........................................S....S..............SSSSK...SSSS...................................S.........S.........K............SSK.K......KS....SK.KKSSSS.SSSSSSSSSS....KSSS..K.K......S....KKK...K...K.....SK.S..............SSSSK...SSSS...K...............................SK.............SSSSSSSSSSSSSSSSSSSSSSSSSSSSS...........S...S..SSS.SSSS.SSSSSSSSSS....SSSSSS....SSS......S.....SSSSSS................................................SSSS.S..............SSSS...SSSS......................................S.........S.........K............SSK.K.......S...SK.SSS.SSSS.SSSSSSSSSS....SSSSSS..K.SSS......S.....SSSSSS..........................................K.....SSSS.S..............SSSS....SSSS...K..................................SK..................SSSSSSSSSSSSSSSSSSSSSSSSSSS............................................................................................................................................................................S.............................. > ........ > ....................................................................................................................................................................................................................................................................................................................................S..........S..........K.................K.K...........K..K...................................K...........................K...............K.................................................................K......S......K.........................................................................................................................................................................................................................................................................................................................K.K.......................................S.K...................SSSSSSSSSSSSSSSSSSSSSSSSSSS............................................ > ........ > ..........................................................................................................................S..........................................................................................................................................................................................................................................................................................................................................................................S..........S..........K.................K.K...........K..K...................................K...........................K.................K.................................................................K......S.................................................................................................................................................................................................................................................................................................... > ........ > ....................K.K.......................................S.K..............SSSSSSSSSSSSSSSSSSSSSSSSSSSSS........S...S..SSS.SSSS.SSSSSSSSSS....SSSSSS..S.SSS......S.....SSSSSS..........................................SS....SSSS.S..............SSSS....SSSS...................................S.........S....................K....SSSSSSSSSSSSSSSSSSSSSSSSSSS.SSS.............SSS....................K.............SSS...................S.SSS............SSSSSS..........................................SS.....SSSS.............................................................................................................................................................................................................................................................................................................................................................SSSSSSSSSSSSSSSSSSSS...............SSSSSSSSSSSSSSSSSSSSSSSSSSS...............SSS...................................SSS................ > .....SSS > .............SSS...SSS..........................................SS....SSSS..............................................................................................................................................................................................................................................................................................................................................................SSSSSSSSSSSSSSSSSSSS.................................................................S...S............................................................................................................................................................................................................................................................................................................................................................................................................................................................................................ > ........ > .........................................................F..F............................................................................................................................................................................................................................................................................................................................................................................................F...........................................F......................................K.K.............................K.....K..K...KK........................SSSSSSSS................................................................................................................................................................................................................................................................................................................................................................................................... > ........ > .......................................................................................................................................................................................................................................................................................................................................................................................................................................................................................K...........K.....................................................................................................................................................................................................................................................................................................................................K......................................K.................................................................................S........................................................... > ........ > .......................................................................................................................................................................................................................................................S..S..............................................................................................................................................................................................................................................................................................................................................................................................................................................................................S.SSS....SSSSSS................................................................................................. > ====================================================================== > ERROR: test_improvement (test_quadpack.TestCtypesQuad) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\numpy\testing\decorators.py", line > 146, in skipper_func > return f(*args, **kwargs) > File > "C:\Python27\lib\site-packages\scipy\integrate\tests\test_quadpack.py", > line 42, in setUp > self.lib = ctypes.CDLL(file) > File "C:\Python27\Lib\ctypes\__init__.py", line 365, in __init__ > self._handle = _dlopen(self._name, mode) > WindowsError: [Error 193] %1 is not a valid Win32 application > > ====================================================================== > ERROR: test_typical (test_quadpack.TestCtypesQuad) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\numpy\testing\decorators.py", line > 146, in skipper_func > return f(*args, **kwargs) > File > "C:\Python27\lib\site-packages\scipy\integrate\tests\test_quadpack.py", > line 42, in setUp > self.lib = ctypes.CDLL(file) > File "C:\Python27\Lib\ctypes\__init__.py", line 365, in __init__ > self._handle = _dlopen(self._name, mode) > WindowsError: [Error 193] %1 is not a valid Win32 application > > ====================================================================== > FAIL: test_interpolate.TestInterp1D.test_circular_refs > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\nose\case.py", line 197, in runTest > self.test(*self.arg) > File > "C:\Python27\lib\site-packages\scipy\interpolate\tests\test_interpolate.py", > line 353, in test_circular_refs > del interp > File "C:\Python27\Lib\contextlib.py", line 24, in __exit__ > self.gen.next() > File "C:\Python27\lib\site-packages\scipy\lib\_gcutils.py", line 96, in > assert_deallocated > raise ReferenceError("Remaining reference(s) to object") > ReferenceError: Remaining reference(s) to object > > ====================================================================== > FAIL: test__gcutils.test_assert_deallocated > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\nose\case.py", line 197, in runTest > self.test(*self.arg) > File "C:\Python27\lib\site-packages\scipy\lib\tests\test__gcutils.py", > line 57, in test_assert_deallocated > del c > File "C:\Python27\Lib\contextlib.py", line 24, in __exit__ > self.gen.next() > File "C:\Python27\lib\site-packages\scipy\lib\_gcutils.py", line 96, in > assert_deallocated > raise ReferenceError("Remaining reference(s) to object") > ReferenceError: Remaining reference(s) to object > > ====================================================================== > FAIL: test_syr_her (test_blas.TestFBLAS2Simple) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\scipy\linalg\tests\test_blas.py", > line 316, in test_syr_her > resz_reverse, rtol=rtol) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 1297, > in assert_allclose > verbose=verbose, header=header) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 665, > in assert_array_compare > raise AssertionError(msg) > AssertionError: > Not equal to tolerance rtol=1e-07, atol=0 > > (mismatch 100.0%) > x: array([[ 0.000000e+00+0.j, 0.000000e+00+0.j, 0.000000e+00+0.j, > 0.000000e+00+0.j], > [ 0.000000e+00+0.j, 0.000000e+00+0.j, 0.000000e+00+0.j,... > y: array([[-15.+112.j, -13. +82.j, -11. +52.j, -9. +22.j], > [ 0. +0.j, -11. +60.j, -9. +38.j, -7. +16.j], > [ 0. +0.j, 0. +0.j, -7. +24.j, -5. +10.j], > [ 0. +0.j, 0. +0.j, 0. +0.j, -3. +4.j]]) > > ====================================================================== > FAIL: test_decomp.test_eigh('general ', 6, 'F', True, False, False, (2, 4)) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\nose\case.py", line 197, in runTest > self.test(*self.arg) > File "C:\Python27\lib\site-packages\scipy\linalg\tests\test_decomp.py", > line 648, in eigenhproblem_general > assert_array_almost_equal(diag2_, ones(diag2_.shape[0]), DIGITS[dtype]) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 842, > in assert_array_almost_equal > precision=decimal) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 665, > in assert_array_compare > raise AssertionError(msg) > AssertionError: > Arrays are not almost equal to 4 decimals > > (mismatch 100.0%) > x: array([ 0., 0., 0.], dtype=float32) > y: array([ 1., 1., 1.]) > > ====================================================================== > FAIL: Tests for the minimize wrapper. > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\nose\case.py", line 197, in runTest > self.test(*self.arg) > File > "C:\Python27\lib\site-packages\scipy\optimize\tests\test_optimize.py", line > 512, in test_minimize > self.test_powell(True) > File > "C:\Python27\lib\site-packages\scipy\optimize\tests\test_optimize.py", line > 241, in test_powell > atol=1e-14, rtol=1e-7) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 1297, > in assert_allclose > verbose=verbose, header=header) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 665, > in assert_array_compare > raise AssertionError(msg) > AssertionError: > Not equal to tolerance rtol=1e-07, atol=1e-14 > > (mismatch 100.0%) > x: array([[ 0.750776, -0.441569, 0.47101 ], > [ 0.750776, -0.441569, 0.480525], > [ 1.501553, -0.883139, 0.951535],... > y: array([[ 0.72949 , -0.441569, 0.47101 ], > [ 0.72949 , -0.441569, 0.480525], > [ 1.45898 , -0.883139, 0.951535],... > > ====================================================================== > FAIL: Powell (direction set) optimization routine > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\nose\case.py", line 197, in runTest > self.test(*self.arg) > File > "C:\Python27\lib\site-packages\scipy\optimize\tests\test_optimize.py", line > 241, in test_powell > atol=1e-14, rtol=1e-7) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 1297, > in assert_allclose > verbose=verbose, header=header) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 665, > in assert_array_compare > raise AssertionError(msg) > AssertionError: > Not equal to tolerance rtol=1e-07, atol=1e-14 > > (mismatch 100.0%) > x: array([[ 0.750776, -0.441569, 0.47101 ], > [ 0.750776, -0.441569, 0.480525], > [ 1.501553, -0.883139, 0.951535],... > y: array([[ 0.72949 , -0.441569, 0.47101 ], > [ 0.72949 , -0.441569, 0.480525], > [ 1.45898 , -0.883139, 0.951535],... > > ====================================================================== > FAIL: test_arpack.test_linearoperator_deallocation > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\nose\case.py", line 197, in runTest > self.test(*self.arg) > File > "C:\Python27\lib\site-packages\scipy\sparse\linalg\eigen\arpack\tests\test_arpack.py", > line 798, in test_linearoperator_deallocation > pass > File "C:\Python27\Lib\contextlib.py", line 24, in __exit__ > self.gen.next() > File "C:\Python27\lib\site-packages\scipy\lib\_gcutils.py", line 96, in > assert_deallocated > raise ReferenceError("Remaining reference(s) to object") > ReferenceError: Remaining reference(s) to object > > ====================================================================== > FAIL: test_beta (test_basic.TestCephes) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\scipy\special\tests\test_basic.py", > line 144, in test_beta > assert_allclose(cephes.beta(0.0342, 171), 24.070498359873497, > rtol=1e-14, atol=0) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 1297, > in assert_allclose > verbose=verbose, header=header) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 665, > in assert_array_compare > raise AssertionError(msg) > AssertionError: > Not equal to tolerance rtol=1e-14, atol=0 > > (mismatch 100.0%) > x: array(24.07049835987154) > y: array(24.070498359873497) > > ====================================================================== > FAIL: test_betaincinv (test_basic.TestCephes) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\scipy\special\tests\test_basic.py", > line 157, in test_betaincinv > assert_allclose(cephes.betaincinv(0.0342, 171, 0.25), > 8.4231316935498957e-21, rtol=1e-12, atol=0) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 1297, > in assert_allclose > verbose=verbose, header=header) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 665, > in assert_array_compare > raise AssertionError(msg) > AssertionError: > Not equal to tolerance rtol=1e-12, atol=0 > > (mismatch 100.0%) > x: array(8.423131693529815e-21) > y: array(8.423131693549896e-21) > > ====================================================================== > FAIL: test_data.test_boost(<Data for binom: > binomial_data_ipp-binomial_data>,) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\nose\case.py", line 197, in runTest > self.test(*self.arg) > File "C:\Python27\lib\site-packages\scipy\special\tests\test_data.py", > line 481, in _test_factory > test.check(dtype=dtype) > File "C:\Python27\lib\site-packages\scipy\special\_testutils.py", line > 292, in check > assert_(False, "\n".join(msg)) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 53, in > assert_ > raise AssertionError(smsg) > AssertionError: > Max |adiff|: 9.70919e+24 > Max |rdiff|: 7.10315e-14 > Bad results (70 out of 159) for the following points (in output 0): > 42.0 21.0 > => 538257874440.0024 != 538257874440.0 > (rdiff 4.422361858108481e-15) > 46.0 21.0 > => 6943526580276.032 != 6943526580276.0 > (rdiff 4.6412384438109286e-15) > 82.0 21.0 => > 1.8330655594514242e+19 != 1.8330655594514647e+19 (rdiff > 2.21216310518291e-14) > 82.0 29.0 => > 1.2576607627509685e+22 != 1.2576607627510093e+22 (rdiff > 3.2516291524073137e-14) > 82.0 33.0 => > 8.999865905487466e+22 != 8.999865905487887e+22 (rdiff > 4.679048843863541e-14) > 82.0 42.0 => > 4.146706622571353e+23 != 4.1467066225715386e+23 (rdiff > 4.482872076557015e-14) > 82.0 46.0 => > 2.322318339533674e+23 != 2.3223183395337372e+23 (rdiff > 2.71635163388777e-14) > 95.0 21.0 => > 6.112305389981363e+20 != 6.112305389981495e+20 (rdiff > 2.165839426429588e-14) > 95.0 29.0 => > 2.146280142106093e+24 != 2.1462801421060996e+24 (rdiff > 3.0016822210721096e-15) > 95.0 33.0 => > 3.7802224438384017e+25 != 3.780222443838485e+25 (rdiff > 2.204165674911803e-14) > 95.0 42.0 => > 1.7198752232706476e+27 != 1.7198752232706825e+27 (rdiff > 2.0297690035618223e-14) > 95.0 46.0 => > 3.086205608690921e+27 != 3.08620560869098e+27 (rdiff > 1.923839025264666e-14) > 122.0 21.0 > => 2.05054879430629e+23 != 2.050548794306221e+23 > (rdiff 3.370908808019204e-14) > 122.0 29.0 => > 9.654999814547143e+27 != 9.65499981454662e+27 (rdiff > 5.4093012244985185e-14) > 122.0 33.0 => > 6.889061799493289e+29 != 6.88906179949298e+29 (rdiff > 4.473977857490729e-14) > 122.0 42.0 => > 9.820488930268602e+32 != 9.820488930268152e+32 (rdiff > 4.5785845286258346e-14) > 122.0 46.0 => > 9.517963588769975e+33 != 9.517963588769497e+33 (rdiff > 5.0148279981909684e-14) > 122.0 82.0 => > 2.546052685625168e+32 != 2.5460526856250765e+32 (rdiff > 3.594314640260528e-14) > 122.0 95.0 > => 8.77923835320522e+26 != 8.779238353204766e+26 > (rdiff 5.1661491374367e-14) > 125.0 21.0 => > 3.577965774451988e+23 != 3.577965774451971e+23 (rdiff > 4.689037586607358e-15) > 125.0 29.0 => > 2.1471697865847835e+28 != 2.1471697865846783e+28 (rdiff > 4.895435483124065e-14) > 125.0 33.0 => > 1.7431114722001235e+30 != 1.7431114722001072e+30 (rdiff > 9.365751364490987e-15) > 125.0 42.0 => > 3.3961976443365834e+33 != 3.396197644336376e+33 (rdiff > 6.110535739146696e-14) > 125.0 46.0 => > 3.824445086978443e+34 != 3.8244450869782213e+34 (rdiff > 5.800111975502932e-14) > 125.0 82.0 => > 6.555451266975118e+33 != 6.555451266974865e+33 (rdiff > 3.851600739996281e-14) > 125.0 95.0 => > 6.870943317071316e+28 != 6.870943317070972e+28 (rdiff > 5.018333445043497e-14) > 135.0 29.0 => > 2.6545984839545678e+29 != 2.654598483954587e+29 (rdiff > 7.289767083732218e-15) > 135.0 33.0 => > 3.2226838569785926e+31 != 3.222683856978583e+31 (rdiff > 2.9346841443966352e-15) > 135.0 42.0 > => 1.65539796634103e+35 != 1.6553979663410087e+35 > (rdiff 1.2814897756371891e-14) > 135.0 46.0 => > 2.9618674395653716e+36 != 2.9618674395653185e+36 (rdiff > 1.7936867201619373e-14) > 135.0 82.0 => > 1.3239687524906834e+38 != 1.3239687524906744e+38 (rdiff > 6.8483063743406465e-15) > 135.0 95.0 => > 3.1921559888457253e+34 != 3.1921559888457077e+34 (rdiff > 5.4898341219098594e-15) > 137.0 21.0 => > 2.891654383984857e+24 != 2.8916543839848387e+24 (rdiff > 6.312514769778831e-15) > 137.0 29.0 => > 4.280069137507773e+29 != 4.28006913750795e+29 (rdiff > 4.126698476389006e-14) > 137.0 33.0 => > 5.605400076850502e+31 != 5.605400076850724e+31 (rdiff > 3.9689909469779893e-14) > 137.0 42.0 => > 3.453905364934577e+35 != 3.453905364934566e+35 (rdiff > 3.2045019404969756e-15) > 137.0 46.0 => > 6.738158013916879e+36 != 6.738158013917095e+36 (rdiff > 3.206340162178697e-14) > 137.0 95.0 => > 3.453905364934577e+35 != 3.453905364934566e+35 (rdiff > 3.2045019404969756e-15) > 143.0 21.0 => > 7.639598299464504e+24 != 7.639598299464654e+24 (rdiff > 1.967693188404081e-14) > 143.0 29.0 => > 1.7138496567724698e+30 != 1.713849656772535e+30 (rdiff > 3.810263889765402e-14) > 143.0 33.0 => > 2.7947962468926867e+32 != 2.7947962468928852e+32 (rdiff > 7.10315364832748e-14) > 143.0 42.0 > => 2.91038363316998e+36 != 2.910383633170114e+36 > (rdiff 4.60410605063329e-14) > 143.0 46.0 => > 7.281844590777859e+37 != 7.281844590778005e+37 (rdiff > 2.010388427601942e-14) > 143.0 95.0 => > 3.0056975544912714e+38 != 3.005697554491347e+38 (rdiff > 2.5138212463529437e-14) > 143.0 122.0 => > 7.639598299464504e+24 != 7.639598299464654e+24 (rdiff > 1.967693188404081e-14) > 144.0 21.0 => > 8.943919960348817e+24 != 8.943919960348863e+24 (rdiff > 5.162266504697016e-15) > 144.0 29.0 => > 2.1460378310892105e+30 != 2.1460378310890872e+30 (rdiff > 5.744821363969209e-14) > 144.0 33.0 => > 3.6256816175907336e+32 != 3.62568161759077e+32 (rdiff > 1.0135852188743159e-14) > 144.0 42.0 => > 4.1087768938873574e+36 != 4.10877689388722e+36 (rdiff > 3.347442009280187e-14) > 144.0 46.0 => > 1.0699853276245263e+38 != 1.0699853276245233e+38 (rdiff > 2.8246317692471727e-15) > 144.0 122.0 > => 5.00046434146803e+25 != 5.000464341467773e+25 > (rdiff 5.136303886238907e-14) > 145.0 21.0 => > 1.045861608266637e+25 != 1.045861608266601e+25 (rdiff > 3.44956971374012e-14) > 145.0 29.0 > => 2.68254728886152e+30 != 2.6825472888613593e+30 > (rdiff 5.980909913173422e-14) > 145.0 33.0 => > 4.693962808488381e+32 != 4.69396280848805e+32 (rdiff > 7.046164831898694e-14) > 145.0 42.0 => > 5.784200481685929e+36 != 5.784200481685892e+36 (rdiff > 6.327294560089766e-15) > 145.0 46.0 => > 1.5671502273289442e+38 != 1.5671502273288471e+38 (rdiff > 6.195440181461709e-14) > 145.0 122.0 => > 3.152466650055975e+26 != 3.1524666500557704e+26 (rdiff > 6.495993880527083e-14) > 145.0 125.0 => > 1.7570475018879216e+24 != 1.7570475018878894e+24 (rdiff > 1.8333172373193666e-14) > 148.0 21.0 > => 1.66081725375304e+25 != 1.6608172537529971e+25 > (rdiff 2.586056525059899e-14) > 148.0 29.0 => > 5.186381531354783e+30 != 5.18638153135483e+30 (rdiff > 8.900597825568916e-15) > 148.0 33.0 => > 1.0064458536532161e+33 != 1.0064458536531621e+33 (rdiff > 5.369707206034169e-14) > 148.0 42.0 => > 1.5872551307291334e+37 != 1.5872551307291024e+37 (rdiff > 1.9487415642240105e-14) > 148.0 122.0 => > 6.418858594896177e+28 != 6.418858594895863e+28 (rdiff > 4.892155143322956e-14) > 148.0 125.0 > => 5.25225250880544e+26 != 5.252252508805427e+26 > (rdiff 2.3550858972873946e-15) > 149.0 21.0 => > 1.9332950844468164e+25 != 1.9332950844468484e+25 (rdiff > 1.655076176038335e-14) > 149.0 29.0 => > 6.439757068098981e+30 != 6.439757068098914e+30 (rdiff > 1.0490146397789524e-14) > 149.0 33.0 => > 1.2927623465028022e+33 != 1.2927623465027686e+33 (rdiff > 2.5974487045134947e-14) > 149.0 42.0 => > 2.2102898549405485e+37 != 2.210289854940526e+37 (rdiff > 1.0255378527439437e-14) > 149.0 122.0 => > 3.542259002368573e+29 != 3.542259002368458e+29 (rdiff > 3.2380764064090125e-14) > 149.0 125.0 => > 3.2607734325499903e+27 != 3.260773432550036e+27 (rdiff > 1.3993530521689755e-14) > > ====================================================================== > FAIL: test_data.test_boost(<Data for gamma: > test_gamma_data_ipp-factorials>,) > ---------------------------------------------------------------------- > Traceback (most recent call last): > File "C:\Python27\lib\site-packages\nose\case.py", line 197, in runTest > self.test(*self.arg) > File "C:\Python27\lib\site-packages\scipy\special\tests\test_data.py", > line 481, in _test_factory > test.check(dtype=dtype) > File "C:\Python27\lib\site-packages\scipy\special\_testutils.py", line > 292, in check > assert_(False, "\n".join(msg)) > File "C:\Python27\lib\site-packages\numpy\testing\utils.py", line 53, in > assert_ > raise AssertionError(smsg) > AssertionError: > Max |adiff|: 3.36416e+140 > Max |rdiff|: 3.56868e-14 > Bad results (35 out of 198) for the following points (in output 0): > 47.0 => 5.502622159812033e+57 > != 5.502622159812089e+57 (rdiff 1.0258520328008188e-14) > 50.0 => 6.082818640342751e+62 > != 6.082818640342675e+62 (rdiff 1.2463859588664233e-14) > 51.0 => 3.0414093201712955e+64 > != 3.0414093201713376e+64 (rdiff 1.3839389152907178e-14) > 52.0 => 1.5511187532873519e+66 > != 1.5511187532873822e+66 (rdiff 1.9537961157045428e-14) > 53.0 => 8.065817517094281e+67 > != 8.065817517094388e+67 (rdiff 1.3210853128030717e-14) > 55.0 => 2.308436973392385e+71 > != 2.308436973392414e+71 (rdiff 1.2533812334944586e-14) > 56.0 => 1.2696403353658055e+73 > != 1.2696403353658276e+73 (rdiff 1.742759977049922e-14) > 57.0 => 7.10998587804854e+74 > != 7.109985878048635e+74 (rdiff 1.3278171253713691e-14) > 59.0 => 2.350561331282903e+78 > != 2.3505613312828785e+78 (rdiff 1.050071233255035e-14) > 60.0 => 1.386831185456872e+80 > != 1.3868311854568984e+80 (rdiff 1.8984338680317015e-14) > 62.0 => 5.075802138772193e+83 > != 5.075802138772248e+83 (rdiff 1.083538910645766e-14) > 67.0 => 5.443449390774354e+92 > != 5.443449390774431e+92 (rdiff 1.403940285109478e-14) > 68.0 => 3.6471110918189365e+94 > != 3.647111091818868e+94 (rdiff 1.869380461041295e-14) > 71.0 => 1.1978571669969762e+100 != > 1.1978571669969892e+100 (rdiff 1.0865971283156417e-14) > 72.0 => 8.504785885678504e+101 > != 8.504785885678623e+101 (rdiff 1.4034165979338454e-14) > 73.0 => 6.123445837688725e+103 != > 6.1234458376886085e+103 (rdiff 1.8972113268364947e-14) > 74.0 => 4.470115461512762e+105 != > 4.4701154615126844e+105 (rdiff 1.7316637068366878e-14) > 79.0 => 1.1324281178206174e+115 != > 1.1324281178206297e+115 (rdiff 1.0816230951910165e-14) > 80.0 => 8.946182130782832e+116 > != 8.946182130782976e+116 (rdiff 1.611677089398549e-14) > 81.0 => 7.156945704626265e+118 > != 7.156945704626381e+118 (rdiff 1.614806559475051e-14) > 83.0 => 4.7536433370127884e+122 > != 4.753643337012842e+122 (rdiff 1.127054006184548e-14) > 84.0 => 3.9455239697205287e+124 > != 3.945523969720659e+124 (rdiff 3.297642089724322e-14) > 85.0 => 3.314240134565268e+126 > != 3.314240134565353e+126 (rdiff 2.5759817183612377e-14) > 87.0 => 2.422709538367351e+130 != > 2.4227095383672734e+130 (rdiff 3.21206322943374e-14) > 89.0 => 1.8548264225739368e+134 != > 1.8548264225739844e+134 (rdiff 2.5668295182660588e-14) > 90.0 => 1.6507955160908254e+136 > != 1.650795516090846e+136 (rdiff 1.2513966038394803e-14) > 91.0 => 1.485715964481777e+138 != > 1.4857159644817615e+138 (rdiff 1.0456113845414324e-14) > 92.0 => 1.3520015276784442e+140 > != 1.352001527678403e+140 (rdiff 3.05102410078959e-14) > 93.0 => 1.243841405464157e+142 != > 1.2438414054641308e+142 (rdiff 2.1115671814606213e-14) > 94.0 => 1.1567725070816173e+144 != > 1.1567725070816416e+144 (rdiff 2.0972887646477295e-14) > 95.0 => 1.0873661566567573e+146 > != 1.087366156656743e+146 (rdiff 1.3055463191484954e-14) > 96.0 => 1.0329978488238868e+148 > != 1.032997848823906e+148 (rdiff 1.8552500324741778e-14) > 97.0 => 9.916779348709327e+149 > != 9.916779348709496e+149 (rdiff 1.7040815113096152e-14) > 98.0 => 9.61927596824809e+151 > != 9.619275968248212e+151 (rdiff 1.2694188843809571e-14) > 99.0 => 9.426890448883584e+153 > != 9.426890448883248e+153 (rdiff 3.568683137742694e-14) > > ---------------------------------------------------------------------- > Ran 17009 tests in 2055.336s > > FAILED (KNOWNFAIL=95, SKIP=1211, errors=2, failures=11) > > >>> >

1 0

Datetime again
by Charles R Harris 29 Jan '15

29 Jan '15

Hi All, I'm playing with the idea of building a simplified datetime class on top of the current numpy implementation. I believe Pandas does something like this, and blaze will (does?) have a simplified version. The reason for the new class would be to have an easier, and hopefully more portable, API that can be implemented in Python, and maybe pushed down into C when things settle. Thoughts? Chuck

6 11

Float view of complex array
by Jens Jørgen Mortensen 28 Jan '15

28 Jan '15

Hi! I have a view of a 2-d complex array that I would like to view as a 2-d float array. This works OK: >>> np.ones((2, 4), complex).view(float) array([[ 1., 0., 1., 0., 1., 0., 1., 0.], [ 1., 0., 1., 0., 1., 0., 1., 0.]]) but this doesn't: >>> np.ones((2, 4), complex)[:, :2].view(float) Traceback (most recent call last): File "<stdin>", line 1, in <module> ValueError: new type not compatible with array. >>> np.__version__ '1.9.0' and I don't understand why. When looking at the memory layout, I think it should be possible. Jens Jørgen

4 7

F2PY cannot see module-scope variables
by Yuxiang Wang 26 Jan '15

26 Jan '15

Dear all, Sorry about being new to both Fortran 90 and f2py. I have a module in fortran, written as follows, with a module-scope variable dp: ======================================== ! testf2py.f90 module testf2py implicit none private public dp, i1 integer, parameter :: dp=kind(0.d0) contains real(dp) function i1(m) real(dp), intent(in) :: m(3, 3) i1 = m(1, 1) + m(2, 2) + m(3, 3) return end function i1 end module testf2py ======================================== Then, if I run f2py -c testf2py.f90 -m testf2py It would report an error, stating that dp was not declared. If I copy the module-scope to the function-scope, it would work. ======================================== ! testf2py.f90 module testf2py implicit none private public i1 integer, parameter :: dp=kind(0.d0) contains real(dp) function i1(m) integer, parameter :: dp=kind(0.d0) real(dp), intent(in) :: m(3, 3) i1 = m(1, 1) + m(2, 2) + m(3, 3) return end function i1 end module testf2py ======================================== However, this does not look like the best coding practice though, as it is pretty "wet". Any ideas? Thanks, Shawn -- Yuxiang "Shawn" Wang Gerling Research Lab University of Virginia yw5aj(a)virginia.edu +1 (434) 284-0836 https://sites.google.com/a/virginia.edu/yw5aj/

2 2

New function `count_unique` to generate contingency tables.
by Warren Weckesser 25 Jan '15

25 Jan '15

I created a pull request (https://github.com/numpy/numpy/pull/4958) that defines the function `count_unique`. `count_unique` generates a contingency table from a collection of sequences. For example, In [7]: x = [1, 1, 1, 1, 2, 2, 2, 2, 2] In [8]: y = [3, 4, 3, 3, 3, 4, 5, 5, 5] In [9]: (xvals, yvals), counts = count_unique(x, y) In [10]: xvals Out[10]: array([1, 2]) In [11]: yvals Out[11]: array([3, 4, 5]) In [12]: counts Out[12]: array([[3, 1, 0], [1, 1, 3]]) It can be interpreted as a multi-argument generalization of `np.unique(x, return_counts=True)`. It overlaps with Pandas' `crosstab`, but I think this is a pretty fundamental counting operation that fits in numpy. Matlab's `crosstab` (http://www.mathworks.com/help/stats/crosstab.html) and R's `table` perform the same calculation (with a few more bells and whistles). For comparison, here's Pandas' `crosstab` (same `x` and `y` as above): In [28]: import pandas as pd In [29]: xs = pd.Series(x) In [30]: ys = pd.Series(y) In [31]: pd.crosstab(xs, ys) Out[31]: col_0 3 4 5 row_0 1 3 1 0 2 1 1 3 And here is R's `table`: > x <- c(1,1,1,1,2,2,2,2,2) > y <- c(3,4,3,3,3,4,5,5,5) > table(x, y) y x 3 4 5 1 3 1 0 2 1 1 3 Is there any interest in adding this (or some variation of it) to numpy? Warren

5 11

What should recfromcsv defaults be?
by Charles R Harris 24 Jan '15

24 Jan '15

Hi All, This question comes apropos this bugfix #5495 <https://github.com/numpy/numpy/pull/5495> that ensures that the default options get passed down the call chain. The current defaults are kwargs.setdefault("case_sensitive", "lower") kwargs.setdefault("names", True) kwargs.setdefault("delimiter", ",") kwargs.setdefault("dtype", None) The ones in question are for "names" and "case_sensitive", that, due to the bug, were defaulting to 'True' and None respectively. I think those defaults should be kept rather than the values currently specified in the recfromcsv definition. However, I don't use these tools, so would like some feedback from those who do. Chuck

1 0