Mailman 3 February 2009 - SciPy-Dev

huge speed regression in loadmat from 0.6.0 to 0.7.0
by Nathaniel Smith Feb. 18, 2009

Feb. 18, 2009

The improvements to loadmat in 0.7.0 are wonderful! Thanks for the work on that. But I've run into one snag... on a matlab file I care about: $ easy_install scipy==0.6.0 $ time python -c 'import scipy.io; scipy.io.loadmat("test.mat")' real 0m4.172s user 0m2.908s sys 0m1.056s $ easy_install scipy==0.7.0 $ time python -c 'import scipy.io; scipy.io.loadmat("test.mat")' real 3m10.556s user 1m14.713s sys 1m55.731s So it became ~50 times slower, and quite unusable. All that time seems to be disappearing into GzipInputStream.__fill, and in particular, line_profiler says: 95 8509 270975 31.8 0.1 data = self.fileobj.read(n_to_fetch) 96 8509 37703 4.4 0.0 self._bytes_read += len(data) 97 8509 27164 3.2 0.0 if data: 98 8509 190425980 22379.4 99.6 self.data += self._unzipper.decompress(data) I'm thinking this is one of those times where the quadratic-time overhead to string append is worth avoiding... -- Nathaniel

6 13

updating the numpy/scipy versions in Linux distros
by Joe Harrington Feb. 18, 2009

Feb. 18, 2009

The versions of our stuff about to go out in Ubuntu 9.04 are: numpy 1.1.1 scipy 0.6.0 Would it be possible to get 1.2.1 and 0.7.0 to go instead? Who takes care of pushing these to the various Linux distros? It would be nice if the Packaging section of Developer Zone on the web site made the packaging process a little more transparent... Say what you do to get a release out and to get it in distribution, who does it, what the timetables tend to look like, etc. If you ask for help there, you're likely to get it. Thanks, --jh--

6 8

Scipy.org seems down
by Gael Varoquaux Feb. 18, 2009

Feb. 18, 2009

scipy.org seems down. conference.scipy.org is up, as well as other servers hosted at Enthought... Gaël

1 0

Re: [SciPy-dev] updating the numpy/scipy versions in Linux distros
by Joe Harrington Feb. 18, 2009

Feb. 18, 2009

Werner Hoch writes: > here are some notes about openSUSE. > ... Werner, could you edit http://scipy.org/Developer_Zone/Distros and put the appropriate info into an entry for OpenSUSE? Thanks! --jh-- Date: Sat, 7 Feb 2009 10:57:33 +0100 From: Werner Hoch <werner.ho(a)gmx.de> Subject: Re: [SciPy-dev] updating the numpy/scipy versions in Linux distros To: SciPy Developers List <scipy-dev(a)scipy.org> Message-ID: <200902071057.34169.werner.ho(a)gmx.de> Content-Type: text/plain; charset="iso-8859-1" Hi all, here are some notes about openSUSE. numpy and scipy are not part of the openSUSE core distribution but several projects in the openSUSE BuildService are packaging and using scipy and numpy. (https://build.opensuse.org/) I've maintained numpy and scipy for a while in the science project. Now both packages are maintained in the Education project. Misc links: http://download.opensuse.org/repositories/Education/ http://download.opensuse.org/repositories/Education/openSUSE_11.1/repodata/… http://download.opensuse.org/repositories/Education/openSUSE_11.1/repodata/… The current version are numpy 1.2.1 scipy 0.6.0 I think as soon as scipy 0.7.0 is out the package will be updated in the Education project. Lars has added some patches to scipy 0.6.0 to fix some compilation warnings in the Education project. I don't know if he has reported them upstream to the scipy project. openSUSE_11.1 has some strict rules about compilation warnings. The compiler warnings are checked after the build some warnings are treated as errors. In my privat project (not published) I've created a test build for scipy 0.7.0rc2. The buildservice complains that scipy 0.7.0rc2 has the following errors: ---------- I: Program is using implicit definitions of special functions. these functions need to use their correct prototypes to allow the lightweight buffer overflow checking to work. - Implicit memory/string functions need #include <string.h>. - Implicit *printf functions need #include <stdio.h>. - Implicit *printf functions need #include <stdio.h>. - Implicit *read* functions need #include <unistd.h>. - Implicit *recv* functions need #include <sys/socket.h>. E: python-scipy implicit-fortify-decl scipy/sparse/linalg/dsolve/SuperLU/SRC/xerbla.c:33 I: Program returns random data in a function E: python-scipy no-return-in-nonvoid-function build/src.linux-x86_64-2.6/scipy/sparse/linalg/isolve/iterative/getbreak.f:74, 54, 34, 14 I: Program causes undefined operation (likely same variable used twiceand post/pre incremented in the same expression). e.g. x = x++; Split it in two operations. E: python-scipy sequence-point scipy/sparse/linalg/dsolve/SuperLU/SRC/cutil.c:243 E: python-scipy sequence-point scipy/sparse/linalg/dsolve/SuperLU/SRC/zutil.c:243 ---------- I'm attaching the patches from Lars (they are against scipy 0.6.0) Maybe someone can review them and integrate them into scipy. Regards Werner

1 0

try_umfpack.py
by Nils Wagner Feb. 16, 2009

Feb. 16, 2009

Hi Robert C., I have updated the URL in try_umfpack.py #defaultURL = 'http://www.cise.ufl.edu/research/sparse/HBformat/' defaultURL = 'http://www.cise.ufl.edu/research/sparse/RB/HB/' ./try_umfpack.py -d bcsstm21.tar.gz ************************************************** url: http://www.cise.ufl.edu/research/sparse/RB/HB/bcsstm21.tar.gz file: /tmp/tmpDwKACc.gz format: triplet reading... Traceback (most recent call last): File "./try_umfpack.py", line 222, in <module> main() File "./try_umfpack.py", line 138, in main mtx = readMatrix( matrixName, options ) File "./try_umfpack.py", line 98, in readMatrix mtx = readMatrix( fd ) File "./try_umfpack.py", line 37, in read_triplet nRow, nCol = map( int, fd.readline().split() ) ValueError: invalid literal for int() with base 10: 'bcsstm21/bcsstm21.rb' Any idea ? Nils

2 1

test failures in current trunk: sparse\linalg\isolve, and special
by josef.pktd＠gmail.com Feb. 16, 2009

Feb. 16, 2009

5 failures, * the ones in special have been reported before (I think) >>> from scipy import special >>> special.yn(301,1) nan * I haven't seen before the failures in sparse\linalg\isolve, * first error is a typo that I corrected in trunk Josef Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import scipy >>> scipy.test() Running unit tests for scipy NumPy version 1.3.0.dev6362 NumPy is installed in C:\Programs\Python25\lib\site-packages\numpy SciPy version 0.8.0.dev5551 SciPy is installed in C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\ scipy-0.8.0.dev5551.win32\Programs\Python25\Lib\site-packages\scipy Python version 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Int el)] nose version 0.10.4 ====================================================================== ERROR: Failure: NameError (global name 'path_candiates' is not defined) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Programs\Python25\lib\site-packages\nose-0.10.4-py2.5.egg\nose\loader .py", line 364, in loadTestsFromName addr.filename, addr.module) File "C:\Programs\Python25\lib\site-packages\nose-0.10.4-py2.5.egg\nose\import er.py", line 39, in importFromPath return self.importFromDir(dir_path, fqname) File "C:\Programs\Python25\lib\site-packages\nose-0.10.4-py2.5.egg\nose\import er.py", line 84, in importFromDir mod = load_module(part_fqname, fh, filename, desc) File "C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\scipy-0.8.0.de v5551.win32\Programs\Python25\Lib\site-packages\scipy\weave\__init__.py", line 9 , in <module> from blitz_tools import blitz File "\Programs\Python25\Lib\site-packages\scipy\weave\blitz_tools.py", line 1 1, in <module> import inline_tools File "C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\scipy-0.8.0.de v5551.win32\Programs\Python25\Lib\site-packages\scipy\weave\inline_tools.py", li ne 15, in <module> function_catalog = catalog.catalog() File "C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\scipy-0.8.0.de v5551.win32\Programs\Python25\Lib\site-packages\scipy\weave\catalog.py", line 35 6, in __init__ sys.path.append(default_dir()) File "C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\scipy-0.8.0.de v5551.win32\Programs\Python25\Lib\site-packages\scipy\weave\catalog.py", line 19 7, in default_dir path_candiates.append(os.path.join(tempfile.gettempdir(), NameError: global name 'path_candiates' is not defined ====================================================================== FAIL: test whether all methods converge ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\scipy-0.8.0.de v5551.win32\Programs\Python25\Lib\site-packages\scipy\sparse\linalg\isolve\tests \test_iterative.py", line 101, in test_convergence assert_equal(info,0) File "\Programs\Python25\Lib\site-packages\numpy\testing\utils.py", line 183, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: -10 DESIRED: 0 ====================================================================== FAIL: test whether maxiter is respected ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\scipy-0.8.0.de v5551.win32\Programs\Python25\Lib\site-packages\scipy\sparse\linalg\isolve\tests \test_iterative.py", line 82, in test_maxiter assert_equal(len(residuals), 3) File "\Programs\Python25\Lib\site-packages\numpy\testing\utils.py", line 183, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: 1 DESIRED: 3 ====================================================================== FAIL: test whether all methods accept a trivial preconditioner ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\scipy-0.8.0.de v5551.win32\Programs\Python25\Lib\site-packages\scipy\sparse\linalg\isolve\tests \test_iterative.py", line 132, in test_precond assert_equal(info,0) File "\Programs\Python25\Lib\site-packages\numpy\testing\utils.py", line 183, in assert_equal raise AssertionError(msg) AssertionError: Items are not equal: ACTUAL: -10 DESIRED: 0 ====================================================================== FAIL: test_iv_cephes_vs_amos (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\scipy-0.8.0.de v5551.win32\Programs\Python25\Lib\site-packages\scipy\special\tests\test_basic.p y", line 1653, in test_iv_cephes_vs_amos self.check_cephes_vs_amos(iv, iv, rtol=1e-8, atol=1e-305) File "C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\scipy-0.8.0.de v5551.win32\Programs\Python25\Lib\site-packages\scipy\special\tests\test_basic.p y", line 1640, in check_cephes_vs_amos assert c2.imag != 0, (v, z) AssertionError: (-100.3, 200.5) ====================================================================== FAIL: test_yv_cephes_vs_amos (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\scipy-0.8.0.de v5551.win32\Programs\Python25\Lib\site-packages\scipy\special\tests\test_basic.p y", line 1650, in test_yv_cephes_vs_amos self.check_cephes_vs_amos(yv, yn, rtol=1e-11, atol=1e-305) File "C:\Josef\_progs\building\scipy\scipy-trunk-new-r5551\dist\scipy-0.8.0.de v5551.win32\Programs\Python25\Lib\site-packages\scipy\special\tests\test_basic.p y", line 1640, in check_cephes_vs_amos assert c2.imag != 0, (v, z) AssertionError: (301, 1.0) ---------------------------------------------------------------------- Ran 3365 tests in 70.313s FAILED (KNOWNFAIL=2, SKIP=32, errors=1, failures=5) <nose.result.TextTestResult run=3365 errors=1 failures=5>

3 3

astrolib, pyfits, pyraf repositories relocated
by Perry Greenfield Feb. 13, 2009

Feb. 13, 2009

As previously announced, we have migrated the astrolib, pyfits, and pyraf repositories (along with the respective trac sites) to a server at STScI. The URLs for the new repositories and trac sites are listed below. All of the previous content of the repositories and trac sites (e.g., history and tickets) has been migrated as well. astrolib: svn: https://www.stsci.edu/svn/ssb/astrolib/ trac: https://www.stsci.edu/trac/ssb/astrolib pyfits: svn: https://www.stsci.edu/svn/ssb/pyfits/ trac: https://www.stsci.edu/trac/ssb/astrolib pyraf: svn: https://www.stsci.edu/svn/ssb/pyraf/ trac: https://www.stsci.edu/trac/ssb/pyraf Those that previously had commit privileges for astrolib should contact sienkiew(a)stsci.edu to get access to the new repository. We would again like to thank Enthought for hosting the repositories for the many years that they have. Perry

1 0

WTFM
by Michael McNeil Forbes Feb. 13, 2009

Feb. 13, 2009

I'd like to register "mforbes" for documentation editing as editor/ reviewer and possibly proofer. Thanks, Michael.

2 1

FAIL: test_yn_zeros, test_ynp_zeros, Negative-order Bessels
by Nils Wagner Feb. 13, 2009

Feb. 13, 2009

Hi all, I do get three failures wrt. to special functions and python2.6 ====================================================================== FAIL: test_yn_zeros (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 1598, in test_yn_zeros 488.98055964441374646], rtol=1e-19) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 38, in assert_tol_equal verbose=verbose, header=header) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/utils.py", line 295, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=1e-19, atol=0 (mismatch 100.0%) x: array([ 450.136, 463.057, 472.807, 481.274, 488.981]) y: array([ 450.136, 463.057, 472.807, 481.274, 488.981]) ====================================================================== FAIL: test_ynp_zeros (test_basic.TestBessel) ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 1604, in test_ynp_zeros assert_tol_equal(yvp(443, ao), 0, atol=1e-15) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 38, in assert_tol_equal verbose=verbose, header=header) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/utils.py", line 295, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=1e-07, atol=1e-15 (mismatch 100.0%) x: array([ 1.239e-10, -8.119e-16, 3.608e-16, 5.898e-16, 1.226e-15]) y: array(0) ====================================================================== FAIL: Negative-order Bessels ---------------------------------------------------------------------- Traceback (most recent call last): File "/home/nwagner/local/lib64/python2.6/site-packages/nose-0.10.4-py2.6.egg/nose/case.py", line 182, in runTest self.test(*self.arg) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 1665, in test_ticket_853 assert_tol_equal(iv(-0.5, 1 ), 1.231200214592967) File "/home/nwagner/local/lib64/python2.6/site-packages/scipy/special/tests/test_basic.py", line 38, in assert_tol_equal verbose=verbose, header=header) File "/home/nwagner/local/lib64/python2.6/site-packages/numpy/testing/utils.py", line 295, in assert_array_compare raise AssertionError(msg) AssertionError: Not equal to tolerance rtol=1e-07, atol=0 (mismatch 100.0%) x: array(0.0) y: array(1.231200214592967) Nils

12 50

astrolib repository and trac site being relocated
by Perry Greenfield Feb. 12, 2009

Feb. 12, 2009

We will be relocating the astrolib svn repository to STScI this Friday (February 13, 2009). When the new repository becomes available, we will have the scipy.org repository and trac site inactivated and post the new new urls for the STScI hosted ones. We will continue to provide commit privileges to the repository to those that have software or changes to contribute at the new site. We are very grateful to Enthought for the many years it has hosted the repository. They provided a great service to STScI and the community by doing so. We will continue to use the scipy.org wiki resources for astropy/astrolib-related information Perry Greenfield

1 0