preamble: the bug here seems to be due to incorrect np.asarray(ctypes.cdouble array) ok -- I tried with a debug build of python and -O0 build of numpy, and the same old valgrind... this time valgrind is silent BUT then python itself says test_simple (mvpa2.tests.test_procrust.ProcrusteanMapperTests) ... XXX undetected error ERROR and nose failure (now I see it) has nothing actually to do with the operation on the object but just get reported to the first line after the function call where I guess a problematic object was created... ====================================================================== ERROR: test_simple (mvpa2.tests.test_procrust.ProcrusteanMapperTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/testing/sweepargs.py", line 67, in do_sweep method(*args_, **kwargs_) File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/testing/sweepargs.py", line 67, in do_sweep method(*args_, **kwargs_) File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/testing/tools.py", line 179, in newfunc return func(*arg, **kwargs) File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/tests/test_procrust.py", line 67, in test_simple pm.train(ds) File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/base/learner.py", line 119, in train result = self._train(ds) File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/mappers/procrustean.py", line 161, in _train print "------------" TypeError: object of type 'float' has no len() that function is fancy only in that it uses ctypes to call a function from the cdll.LoadLibrary('liblapack.so'): https://github.com/PyMVPA/PyMVPA/blob/HEAD/mvpa2/support/lapack_svd.py but even if I comment out that call to lapacklib. it still screws up the same way -- so it has to do with those variable definitions before then I guess... ok -- it boils down to numpy.asarray(s) in return statement... I improved it with a printout now where I assigned constructed array to a variable first NB if I swap print lines, it would lead me to crash above, but with this order -- it manages to continue without crashing BUT showing incorrect values. s_arr = numpy.asarray(s) print "s_arr", s_arr print "s:", s return vt, s_arr, u so it gives me $> MVPA_SEED=1928295852 `which nosetests` -s -v mvpa2/tests/test_procrust.py T: MVPA_SEED=1928295852 test_simple (mvpa2.tests.test_procrust.ProcrusteanMapperTests) ... VERSION 1.7.0rc1.dev-ea23de8 s_arr [[ 6.90689888e-310 1.83759219e-316] [ -3.16388621e+134 -3.16388621e+134]] s: <mvpa2.support.lapack_svd.c_double_Array_2 object at 0x28c4ae0> FAIL ====================================================================== FAIL: test_simple (mvpa2.tests.test_procrust.ProcrusteanMapperTests) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/testing/tools.py", line 179, in newfunc return func(*arg, **kwargs) File "/home/yoh/proj/pymvpa/pymvpa/mvpa2/tests/test_procrust.py", line 93, in test_simple "perfectly. Now got d scale*R=%g" % dsR) AssertionError: Single scenario lead to failures of unittest test_simple: on oblique=False : Single scenario lead to failures of unittest test_simple: on svd=dgesvd : We should have got reconstructed rotation+scaling perfectly. Now got d scale*R=1.34232e+134 ---------------------------------------------------------------------- Ran 1 test in 0.222s FAILED (failures=1) [161643 refs] when with 1.6.2 (using the same seed, so if numpy's RNG didn't change -- data should be the same): $> MVPA_SEED=1928295852 nosetests -s -v mvpa2/tests/test_procrust.py T: MVPA_SEED=1928295852 test_simple (mvpa2.tests.test_procrust.ProcrusteanMapperTests) ... VERSION 1.6.2 s_arr [0.775771814652 0.224228185348] s: <mvpa2.support.lapack_svd.c_double_Array_2 object at 0xad65d40> s_arr [0.775771814652 0.224228185348] s: <mvpa2.support.lapack_svd.c_double_Array_2 object at 0xad65c20> s_arr [0.365993114976 0.101324191354 0.0959100894799 0.0861936270658 0.0712098129231 0.0694159685405 0.0636016740376 0.056058975141 0.0481853714064 0.042107175076] s: <mvpa2.support.lapack_svd.c_double_Array_10 object at 0xad65d40> s_arr [0.365993114976 0.101324191354 0.0959100894799 0.0861936270658 0.0712098129231 0.0694159685405 0.0636016740376 0.056058975141 0.0481853714064 0.042107175076] s: <mvpa2.support.lapack_svd.c_double_Array_10 object at 0xad65c20> s_arr [0.775771814652 0.224228185348 0.0] s: <mvpa2.support.lapack_svd.c_double_Array_3 object at 0xad65d40> s_arr [0.775771814652 0.224228185348 0.0] s: <mvpa2.support.lapack_svd.c_double_Array_3 object at 0xad65c20> s_arr [0.703907195999 0.200251541932 0.0] s: <mvpa2.support.lapack_svd.c_double_Array_3 object at 0xad65d40> s_arr [0.703907195999 0.200251541932 0.0] s: <mvpa2.support.lapack_svd.c_double_Array_3 object at 0xad65c20> ok ---------------------------------------------------------------------- Ran 1 test in 7.031s OK MVPA_SEED=1928295852 nosetests -s -v mvpa2/tests/test_procrust.py 7,14s user 0,79s system 96% cpu 8,219 total which immediately shows that np.asarray created a 2d array whenever it should have been a 1d (original definition of s is s=(c_double*min(x,y))()) ok -- added printing of dtype of that array -- with 1.6.2 it is reported as object while with 1.7.0b1 -- float64... from here I pass it onto experts! ;) On Thu, 06 Sep 2012, Yaroslav Halchenko wrote:
On Thu, 06 Sep 2012, Aron Ahmadia wrote:
Are you running the valgrind test with the Python suppression file:�[1]http://svn.python.org/projects/python/trunk/Misc/valgrind-python.supp
yes -- on Debian there is /usr/lib/valgrind/python.supp which comes with python package and I believe enabled by default, and it is identical to above (just dynamic library versions different) but it still produces lots of false positives -- IIRC it needs additional tune ups per architecture etc... I just ignored those messages "manually" and listed the relevant one which comes from numpy functionality. -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik