Mailman 3 October 2008 - SciPy-Dev

tickets 747, 748, 749 can be closed
by Anne Archibald Oct. 9, 2008

Oct. 9, 2008

Tickets 747, 748, 749 can be closed, but unfortunately I (peridot) don't seem to have permission to close them, in spite of having been the one who submitted them. I did put a "closes tickets ..." in the svn commit message, but I don't think there's an automatic ticket-closing gizmo. Anne

2 1

Fwd: tests and partial patch for scipy.stats.distributions
by josef.pktd＠gmail.com Oct. 6, 2008

Oct. 6, 2008

removed patch because of file size, mail bounced ---------- Forwarded message ---------- From: josef.pktd(a)gmail.com Date: Mon, 29 Sep 2008 00:00:25 -0400 Subject: tests and partial patch for scipy.stats.distributions To: scipy-dev(a)scipy.org To help in bug hunting and patching of the discrete distribution in scipy.stats, I wrote several test scripts that could be incorporated in scipy tests. The attachment contains the corrected scipy.stats.distributions.py file and 2 test files. The tests check all discrete distributions at predefined (not randomly chosen) values. Some distributions have some wrong results for some other range of parameters. To try out the patch, it is possible just to backup the current scipy.stats.distributions.py and copy the patched version in its place. Right now, I still left many comments (and some commented out failed attempts for fixing some persistent bugs). In direct comparison with the current trunk file this might help in understanding what the underlying bug is. Since there are a large number of bug fixes in this patch, if any of them get incorporated in scipy, then they can be selectively copied. Many bugs only show up once other bugs are fixed. In terms of importance and sequence the bug fix categories are roughly: * nin correction for vectorize with *args: without this, some methods raise exceptions, e.g. _ppf (I think after many shaky attempts, I found the correct solution) * _ppf did not handle infinite support properly and requires a .tolist in call to ``place`` (no idea why, but this works) * _drv2_moment(self, n, *args) was completely broken, it didn't even have a return and is the base for generic moment calculations individual fixes: * recipinvgauss_gen: rvs added, didn't work before * def moment: for 2nd moment returned the mean (both discrete and continuous distribution * randint: -1 missing in _ppf, definitions differs from those in description pdf * dlaplace: _ppf gives wrong results in tests, replace with generic calculation (Note: the first vectorize nin correction in continuous distributions is still at the wrong spot. I was only looking at discrete distributions in the last few days, since I figured out how vectorize nin works) If you want a cleaner patch or more information, I can provide it next week. Because, I never used numpy/scipy heavily, it took me a long time to figure out what is going on, but it should be more obvious to an expert. But overall my impression is, that using features in numpy.stats.distribution, that are not commonly used, is pretty risky, and until recently didn't have any stringent tests and does not have any reasonably large test coverage. For serious work, I would want to make sure that the results are actually correct. There are still several things that are not covered, e.g. handling of loc and scale, other methods, correctness over full range of parameters. I haven't checked the continuous distributions since the initial fuzz tests, but the test coverage for all methods looks pretty low. Josef Test f Basic Properties ---------------------- The first test just tests for basic properties and their consistency, e.g. cdf, pmf, ppf, stats and moments and the internal methods e.g _ppf. The tests check quite a bit for internal consistency (private methods) since I used the tests for debugging and trying out fixes. >python C:\Programs\Python25\Scripts\nosetests-script.py -s test_discrete_basic.py ... with current trunk, I get for this test: Ran 112 tests in 0.140s FAILED (errors=41, failures=15) after the bugfixes, I get: Ran 112 tests in 8.672s FAILED (failures=5) some failures in binom, dlaplace, zipf are still remaining Chisquare Test -------------- the second test is a chisquare test for the random variables to be close to the theoretical distribution as defined by .cdf with current trunk I get for this test: >python C:\Programs\Python25\Scripts\nosetests-script.py -s test_ndiscrete.py Ran 12 tests in 0.453s FAILED (errors=5) after patching scipy.stats.distribution the only test failure left is in logser, where I think the numpy.random random numbers are wrong >python C:\Programs\Python25\Scripts\nosetests-script.py -s test_ndiscrete.py bernoulli. binom. boltzmann. dlaplace. geom. hypergeom. logserF nbinom. planck. poisson. randint. zipf. ====================================================================== FAIL: test_ndiscrete.test_discrete_rvscdf('logser', (0.59999999999999998,)) ---------------------------------------------------------------------- Traceback (most recent call last): File "c:\programs\python25\lib\site-packages\nose-0.10.3-py2.5.egg\nose\case.p y", line 182, in runTest self.test(*self.arg) File "C:\Josef\work-oth\sort\pypi\test_ndiscrete.py", line 76, in checkchisqua re_discrete assert (pval > alpha), 'chisquare - test for %s at arg = %s' % (distname,str (arg)) AssertionError: chisquare - test for logser at arg = (0.59999999999999998,) ---------------------------------------------------------------------- Ran 12 tests in 19.781s FAILED (failures=1) scipy.stats tests: no failures ----------------------------------------- running the current trunk test for scipy.stats on the pre- and post patch distributions.py gives no errors: >>> from scipy import stats >>> stats.test() Running unit tests for scipy.stats NumPy version 1.2.0rc2 NumPy is installed in C:\Programs\Python25\lib\site-packages\numpy SciPy version 0.7.0.dev SciPy is installed in C:\Josef\_progs\virtualpy25\envscipy\lib\site-packages\sci py Python version 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Int el)] nose version 0.10.3 ................................................................................ C:\Josef\_progs\virtualpy25\envscipy\lib\site-packages\scipy\stats\morestats.py: 618: UserWarning: Ties preclude use of exact statistic. warnings.warn("Ties preclude use of exact statistic.") ............................................................C:\Programs\Python25 \lib\site-packages\numpy\lib\function_base.py:343: Warning: The semantics of histogram has been modified in the current release to fix long-standing issues with outliers handling. The main changes concern 1. the definition of the bin edges, now including the rightmost edge, and 2. the handling of upper outliers, now ignored rather than tallied in the rightmost bin. The previous behaviour is still accessible using `new=False`, but is scheduled to be deprecated in the next release (1.3). *This warning will not printed in the 1.3 release.* Use `new=True` to bypass this warning. Please read the docstring for more information. """, Warning) ................................................................................ ............. ---------------------------------------------------------------------- Ran 233 tests in 2.266s OK <nose.result.TextTestResult run=233 errors=0 failures=0>

2 3

PyArray_FromDims and friends
by Zachary Pincus Oct. 6, 2008

Oct. 6, 2008

Hi all, In testing out svn scipy and numpy, I noticed some run-time errors from scipy.interpolate because the _fitpack module's c sources use PyArray_FromDims and PyArray_FromDimsAndData, which are now deprecated in numpy svn. I opened a ticket and made a patch for this particular case, but I'm not sure if there's an overall strategy (someone writes a very good regexp, say, and lets it loose), or if these will be fixed piecemeal. If the latter, here's the patch: http://scipy.org/scipy/scipy/ticket/723 Zach

7 9

recent scipy.io
by Nils Wagner Oct. 5, 2008

Oct. 5, 2008

====================================================================== ERROR: Failure: ImportError (No module named byteordercodes) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/loader.py", line 364, in loadTestsFromName addr.filename, addr.module) File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/importer.py", line 39, in importFromPath return self.importFromDir(dir_path, fqname) File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/importer.py", line 84, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_byteordercodes.py", line 9, in <module> import scipy.io.byteordercodes as sibc ImportError: No module named byteordercodes ====================================================================== FAIL: test_mio.test_load('double', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testdouble_6.1_SOL2.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testdouble_7.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testdouble_6.5.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testdouble_4.2c_SOL2.mat'], {'testdouble': array([[ 0. ], ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 57, in _check_level assert_array_almost_equal(actual, expected, err_msg=label, decimal=5) File "/usr/local/lib64/python2.5/site-packages/numpy/testing/utils.py", line 310, in assert_array_almost_equal header='Arrays are not almost equal') File "/usr/local/lib64/python2.5/site-packages/numpy/testing/utils.py", line 260, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal test double; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testdouble_6.1_SOL2.mat, variable testdouble (shapes (1, 9), (9, 1) mismatch) x: array([[ 0. , 0.78539816, 1.57079633, 2.35619449, 3.14159265, 3.92699082, 4.71238898, 5.49778714, 6.28318531]]) y: array([[ 0. ], [ 0.78539816], [ 1.57079633],... ====================================================================== FAIL: test_mio.test_load('string', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststring_4.2c_SOL2.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststring_6.5.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststring_7.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststring_6.1_SOL2.mat'], {'teststring': u'"Do nine men interpret?" "Nine men," I nod.'}) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 60, in _check_level "Types %s and %s do not match at %s" % (typex, typac, label) AssertionError: Types <type 'unicode'> and <type 'numpy.ndarray'> do not match at test string; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststring_4.2c_SOL2.mat, variable teststring ====================================================================== FAIL: test_mio.test_load('complex', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcomplex_4.2c_SOL2.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcomplex_6.1_SOL2.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcomplex_6.5.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcomplex_7.1_GLNX86.mat'], {'testcomplex': array([[ 1.000e+00 +0.000e+00j], ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 57, in _check_level assert_array_almost_equal(actual, expected, err_msg=label, decimal=5) File "/usr/local/lib64/python2.5/site-packages/numpy/testing/utils.py", line 310, in assert_array_almost_equal header='Arrays are not almost equal') File "/usr/local/lib64/python2.5/site-packages/numpy/testing/utils.py", line 260, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal test complex; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcomplex_4.2c_SOL2.mat, variable testcomplex (shapes (1, 9), (9, 1) mismatch) x: array([[ 1.00000000e+00 +0.00000000e+00j, 7.07106781e-01 +7.07106781e-01j, 6.12323400e-17 +1.00000000e+00j,... y: array([[ 1.00000000e+00 +0.00000000e+00j], [ 7.07106781e-01 +7.07106781e-01j], [ 6.12323400e-17 +1.00000000e+00j],... ====================================================================== FAIL: test_mio.test_load('multi', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testmulti_4.2c_SOL2.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testmulti_7.1_GLNX86.mat'], {'a': array([[ 1., 2., 3., 4., 5.], ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 57, in _check_level assert_array_almost_equal(actual, expected, err_msg=label, decimal=5) File "/usr/local/lib64/python2.5/site-packages/numpy/testing/utils.py", line 310, in assert_array_almost_equal header='Arrays are not almost equal') File "/usr/local/lib64/python2.5/site-packages/numpy/testing/utils.py", line 260, in assert_array_compare raise AssertionError(msg) AssertionError: Arrays are not almost equal test multi; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testmulti_4.2c_SOL2.mat, variable theta (shapes (1, 9), (9, 1) mismatch) x: array([[ 0. , 0.78539816, 1.57079633, 2.35619449, 3.14159265, 3.92699082, 4.71238898, 5.49778714, 6.28318531]]) y: array([[ 0. ], [ 0.78539816], [ 1.57079633],... ====================================================================== FAIL: test_mio.test_load('onechar', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testonechar_4.2c_SOL2.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testonechar_7.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testonechar_6.1_SOL2.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testonechar_6.5.1_GLNX86.mat'], {'testonechar': u'r'}) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 60, in _check_level "Types %s and %s do not match at %s" % (typex, typac, label) AssertionError: Types <type 'unicode'> and <type 'numpy.ndarray'> do not match at test onechar; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testonechar_4.2c_SOL2.mat, variable testonechar ====================================================================== FAIL: test_mio.test_load('cell', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcell_6.1_SOL2.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcell_7.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcell_6.5.1_GLNX86.mat'], {'testcell': array([This cell contains this string and 3 arrays of increasing length, 1, [1 2], [1 2 3]], dtype=object)}) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 27, in _check_level assert len(expected) == len(actual), "Different list lengths at %s" % label AssertionError: Different list lengths at test cell; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcell_6.1_SOL2.mat, variable testcell ====================================================================== FAIL: test_mio.test_load('emptycell', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testemptycell_7.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testemptycell_6.5.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testemptycell_5.3_SOL2.mat'], {'testemptycell': array([1, 2, [], [], 3], dtype=object)}) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 27, in _check_level assert len(expected) == len(actual), "Different list lengths at %s" % label AssertionError: Different list lengths at test emptycell; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testemptycell_7.1_GLNX86.mat, variable testemptycell ====================================================================== FAIL: test_mio.test_load('stringarray', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststringarray_6.1_SOL2.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststringarray_7.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststringarray_6.5.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststringarray_4.2c_SOL2.mat'], {'teststringarray': array([one , two , three], dtype=object)}) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 30, in _check_level _check_level(level_label, ev, actual[i]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 60, in _check_level "Types %s and %s do not match at %s" % (typex, typac, label) AssertionError: Types <type 'unicode'> and <type 'numpy.unicode_'> do not match at test stringarray; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststringarray_6.1_SOL2.mat, variable teststringarray, [0], ====================================================================== FAIL: test_mio.test_load('struct', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststruct_6.5.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststruct_7.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststruct_6.1_SOL2.mat'], {'teststruct': array([ (u'Rats live on no evil star.', [1.4142135623730951, 2.7182818284590451, 3.1415926535897931], [(1.4142135623730951+1.4142135623730951j), (2.7182818284590451+2.7182818284590451j), (3.1415926535897931+3.1415926535897931j)])], ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 30, in _check_level _check_level(level_label, ev, actual[i]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 60, in _check_level "Types %s and %s do not match at %s" % (typex, typac, label) AssertionError: Types <type 'numpy.void'> and <type 'numpy.ndarray'> do not match at test struct; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststruct_6.5.1_GLNX86.mat, variable teststruct, [0], ====================================================================== FAIL: test_mio.test_load('cellnest', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcellnest_6.5.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcellnest_7.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcellnest_6.1_SOL2.mat'], {'testcellnest': array([1, [2 3 [4 5]]], dtype=object)}) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 27, in _check_level assert len(expected) == len(actual), "Different list lengths at %s" % label AssertionError: Different list lengths at test cellnest; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testcellnest_6.5.1_GLNX86.mat, variable testcellnest ====================================================================== FAIL: test_mio.test_load('structnest', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststructnest_7.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststructnest_6.5.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststructnest_6.1_SOL2.mat'], {'teststructnest': array([[(1, [[(u'number 3',)]])]], ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 30, in _check_level _check_level(level_label, ev, actual[i]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 30, in _check_level _check_level(level_label, ev, actual[i]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 60, in _check_level "Types %s and %s do not match at %s" % (typex, typac, label) AssertionError: Types <type 'numpy.void'> and <class 'scipy.io.matlab.mio5.mat_struct'> do not match at test structnest; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststructnest_7.1_GLNX86.mat, variable teststructnest, [0], , [0], ====================================================================== FAIL: test_mio.test_load('structarr', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststructarr_6.1_SOL2.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststructarr_6.5.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststructarr_7.1_GLNX86.mat'], {'teststructarr': array([[(1, 2)], ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 27, in _check_level assert len(expected) == len(actual), "Different list lengths at %s" % label AssertionError: Different list lengths at test structarr; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/teststructarr_6.1_SOL2.mat, variable teststructarr ====================================================================== FAIL: test_mio.test_load('object', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testobject_7.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testobject_6.5.1_GLNX86.mat', '/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testobject_6.1_SOL2.mat'], {'testobject': <scipy.io.matlab.mio5.MatlabObject instance at 0x33bbcf8>}) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 41, in _check_level assert k in ac_fields, "Missing property at %s" % label AssertionError: Missing property at test object; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testobject_7.1_GLNX86.mat, variable testobject ====================================================================== FAIL: test_mio.test_load('unicode', ['/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testunicode_7.1_GLNX86.mat'], {'testunicode': u'Japanese: \n\u3059\u3079\u3066\u306e\u4eba\u9593\u306f\u3001\u751f\u307e\u308c\u306a\u304c\u3089\u306b\u3057\u3066\u81ea\u7531\u3067\u3042\u308a\u3001\n\u304b\u3064\u3001\u5c0a\u53b3\u3068\u6a29\u5229\u3068 \u306b\u3064\u3044\u3066\u5e73\u7b49\u3067\u3042\u308b\u3002\n\u4eba\u9593\u306f\u3001\u7406\u6027\u3068\u826f\u5fc3\u3068\u3092\u6388\u3051\u3089\u308c\u3066\u304a\u308a\u3001\n\u4e92\u3044\u306b\u540c\u80de\u306e\u7cbe\u795e\u3092\u3082\u3063\u3066\u884c\u52d5\u3057\u306a\u3051\u308c\u3070\u306a\u3089\u306a\u3044\u3002'}) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 60, in _check_level "Types %s and %s do not match at %s" % (typex, typac, label) AssertionError: Types <type 'unicode'> and <type 'numpy.ndarray'> do not match at test unicode; file /usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/data/testunicode_7.1_GLNX86.mat, variable testunicode ====================================================================== FAIL: test_mio.test_round_trip('string_round_trip', {'teststring': u'"Do nine men interpret?" "Nine men," I nod.'}, '4') ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 77, in _rt_check_case _check_case(name, [mat_stream], expected, struct_as_record=True) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 60, in _check_level "Types %s and %s do not match at %s" % (typex, typac, label) AssertionError: Types <type 'unicode'> and <type 'numpy.ndarray'> do not match at test string_round_trip; file <cStringIO.StringO object at 0x3751c00>, variable teststring ====================================================================== FAIL: test_mio.test_round_trip('onechar_round_trip', {'testonechar': u'r'}, '4') ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/nose-0.10.1-py2.5.egg/nose/case.py", line 203, in runTest self.test(*self.arg) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 77, in _rt_check_case _check_case(name, [mat_stream], expected, struct_as_record=True) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 70, in _check_case _check_level(k_label, expected, matdict[k]) File "/usr/local/lib64/python2.5/site-packages/scipy/io/matlab/tests/test_mio.py", line 60, in _check_level "Types %s and %s do not match at %s" % (typex, typac, label) AssertionError: Types <type 'unicode'> and <type 'numpy.ndarray'> do not match at test onechar_round_trip; file <cStringIO.StringO object at 0x3751810>, variable testonechar ====================================================================== FAIL: test_imresize (test_pilutil.TestPILUtil) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib64/python2.5/site-packages/numpy/testing/decorators.py", line 82, in skipper return f(*args, **kwargs) File "/usr/local/lib64/python2.5/site-packages/scipy/misc/tests/test_pilutil.py", line 24, in test_imresize assert_equal(im1.shape,(11,22)) File "/usr/local/lib64/python2.5/site-packages/numpy/testing/utils.py", line 174, in assert_equal assert_equal(len(actual),len(desired),err_msg,verbose) File "/usr/local/lib64/python2.5/site-packages/numpy/testing/utils.py", line 183, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 0 DESIRED: 2 ---------------------------------------------------------------------- Ran 2342 tests in 36.958s FAILED (failures=17, errors=1)

2 1

Using nose
by Anne Archibald Oct. 5, 2008

Oct. 5, 2008

Hi, I have a particular pattern of tests I'd like to implement, and I don't know a good way to make nose carry it out. The pattern is this: I have a class with a setUp function that creates various variables (an array of test data, etc.). I also have a handful of test_whatever methods. This works fine. But now I'd like to apply exactly the same test methods to a different set of data. My first attempt was to simply create a derived class which overrode the setUp method, but unfortunately neither unittest nor nose will run tests defined in the parent class. Any suggestions? Thanks, Anne

5 7

[patch] read/write v5 .mat files with structs, cell arrays, objects, or function handles
by Thouis (Ray) Jones Oct. 5, 2008

Oct. 5, 2008

I just submitted a patch (http://scipy.org/scipy/scipy/ticket/743) extending the functionality of io.matlab for v5 files, adding writers for structs, cell arrays, & objects, and readers+writers for function handles and 64-bit ints. The patch modifies the numpy types of strings, structs, and objects loaded from matlab: matlab strings are loaded as numpy arrays of unicode strings, rather than numpy arrays of objects. structs are loaded as numpy arrays with dtype [(field_name, object), ...] matlab objects are wrapped in an mio5.MatlabObject instance, which contains a 1x1 struct & the object classname. (function handles are also wrapped in a special object in mio5.py) These changes were to make the scipy<->matlab mapping more explicit. The changes to the string type probably don't break any existing code. The changes to the struct & matlab objects probably do. It's possible to make both of these backward compatible, if that's a priority. We're looking for feedback on that (or any other) issue. Ray Jones

7 14

Question about Kaiser Implementation in firwin
by Buehler, Eric (AGRE) Oct. 4, 2008

Oct. 4, 2008

Hello, I have been using the signal.firwin FIR design tool and I have a question about the implementation. Scipy-0.4.8 Numpy-0.9.6 I am generating a kaiser window with the following parameters: N = 128 Cutoff = ~4e7 width = .3 All of the special kaiser generation parameters in the function follow Oppenheim and Schafer well. Even the sinc function for linear phase is follows exactly. However, the last line of the firwin function is causing me some heartburn. filter_design.py 1538 win = get_window(window,N,fftbins=1) 1539 alpha = N//2 1540 m = numpy.arange(0,N) 1541 h = win*special.sinc(cutoff*(m-alpha)) 1542 return h / sum(h) Line 1542 of filter_design.py, "return h / sum(h)", normalizes the function where it doesn't seem necessary, at least in the kaiser window case. Without the normalization, the kaiser window already returns a value of 1 at the zero frequency point. This normalization scales all of the data, making the window difficult to use in the frequency domain. Can someone point me to the rationale for this line? Looking at the code, this seems to be a pretty recent change (within the last year/year and a half). Thanks, Eric Buehler eric <dot> buehler <at> smiths-aerospace <dot> com ****************************************** The information contained in, or attached to, this e-mail, may contain confidential information and is intended solely for the use of the individual or entity to whom they are addressed and may be subject to legal privilege. If you have received this e-mail in error you should notify the sender immediately by reply e-mail, delete the message from your system and notify your system manager. Please do not copy it for any purpose, or disclose its contents to any other person. The views or opinions presented in this e-mail are solely those of the author and do not necessarily represent those of the company. The recipient should check this e-mail and any attachments for the presence of viruses. The company accepts no liability for any damage caused, directly or indirectly, by any virus transmitted in this email. ******************************************

3 2

Next scipy release 0.7
by Nils Wagner Oct. 3, 2008

Oct. 3, 2008

Hi Jarrod, Just curious. Is there a new deadline for scipy 0.7 ? Cheers, Nils

7 12

Add density of water to constants?
by Ryan May Oct. 2, 2008

Oct. 2, 2008

Hi, I was just cleaning up my own hand-rolled constants file and noticed that the only one not already in scipy's own constants file was the density of liquid water. I'm not sure how much we want to accumulate more of these physical constants in there, but here's a patch nonetheless. Ryan -- Ryan May Graduate Research Assistant School of Meteorology University of Oklahoma --- constants/constants.py.old 2008-10-02 12:03:38.000000000 -0500 +++ constants/constants.py 2008-10-02 12:06:53.000000000 -0500 @@ -70,6 +70,7 @@ sigma = Stefan_Bolzmann = _cd('Stefan-Boltzmann constant') Wien = _cd('Wien displacement law constant') Rydberg = _cd('Rydberg constant') +density_liquid_water = 1000. #defined, for kg m^-3 #weight in kg gram = 1e-3

2 2

Proposal: scipy.spatial
by Anne Archibald Oct. 1, 2008

Oct. 1, 2008

Hi, Once again there has been a thread on the numpy/scipy mailing lists requesting (essentially) some form of spatial data structure. Pointers have been posted to ANN (sadly LGPLed and in C++) as well as a handful of pure-python implementations of kd-trees. I suggest the creation of a new submodule of scipy, scipy.spatial, to contain spatial data structures and algorithms. Specifically, I propose it contain a kd-tree implementation providing nearest-neighbor, approximate nearest-neighbor, and all-points-near queries. I have a few other suggestions for things it might contain, but kd-trees seem like a good first step. 2008/9/27 Nathan Bell <wnbell(a)gmail.com>: > On Sat, Sep 27, 2008 at 11:18 PM, Anne Archibald > <peridot.faceted(a)gmail.com> wrote: >> >> I think a kd-tree implementation would be a valuable addition to >> scipy, perhaps in a submodule scipy.spatial that might eventually >> contain other spatial data structures and algorithms. What do you >> think? Should we have one? Should it be based on Sturla Molden's code, >> if the license permits? I am willing to contribute one, if not. > > +1 Judging that your vote and mine are enough in the absence of dissenting voices, I have written an implementation based on yours, Sturla Molden's, and the algorithms described by the authors of the ANN library. Before integrating it into scipy, though, I'd like to send it around for comments. Particular issues: * It's pure python right now, but with some effort it could be partially or completely cythonized. This is probably a good idea in the long run. In the meantime I have crudely vectorized it so that users can at least avoid looping in their own code. * It is able to return the r nearest neighbors, optionally subject to a maximum distance limit, as well as approximate nearest neighbors. * It is not currently set up to conveniently return all points within some fixed distance of the target point, but this could easily be added. * It returns distances and indices of nearest neighbors in the original array. * The testing code is, frankly, a mess. I need to look into using nose in a sensible fashion. * The license is the scipy license. I am particularly concerned about providing a convenient return format. The natural return from a query is a list of neighbors, since it may have variable length (there may not be r elements in the tree, or you might have supplied a maximum distance which doesn't contain r points). For a single query, it's simple to return a python list (should it be sorted? currently it's a heap). But if you want to vectorize the process, storing the results in an array becomes cumbersome. One option is an object array full of lists; another, which, I currently use, is an array with nonexistent points marked with an infinite distance and an invalid index. A third would be to return masked arrays. How do you recommend handling this variable (or potentially-variable) sized output? > If you're implementing one, I would highly recommend the > "left-balanced" kd-tree. > http://www2.imm.dtu.dk/pubdb/views/edoc_download.php/2535/pdf/imm2535.pdf Research suggests that at least in high dimension a more geometric balancing criterion can produce vastly better results. I used the "sliding midpoint" rule, which doesn't allow such a numerical balancing but does ensure that you don't have too many long skinny cells (since a sphere tends to cut very many of these). I've also been thinking about what else would go in scipy.spatial. I think it would be valuable to have a reasonably efficient distance matrix function (we seem to get that question a lot, and the answer's not trivial) as well as a sparse distance matrix function based on the kd-trees. The module could potentially also swallow the (currently sandboxed?) delaunay code. Anne

6 9