Mailman 3 April 2002 - SciPy-Dev

Linalg2 benchmarks
by Jochen Küpper April 7, 2002

April 7, 2002

Travis showed really impressive numbers for linal2. Here is what I get -- less impressive, still ok? ,---- | >>> scipy.linalg2.basic.test() | ................................ | Finding matrix determinant | ================================== | | contiguous | non-contiguous | ---------------------------------------------- | size | scipy | Numeric | scipy | Numeric | 20 | 0.35 | 0.75 | 0.35 | 0.87 (secs for 2000 calls) | 100 | 0.82 | 1.43 | 0.83 | 1.84 (secs for 300 calls) | 500 | 0.89 | 1.09 | 0.87 | 1.29 (secs for 4 calls) | . | Solving system of linear equations | ================================== | | contiguous | non-contiguous | ---------------------------------------------- | size | scipy | Numeric | scipy | Numeric | 20 | 0.53 | 0.65 | 0.53 | 0.77 (secs for 2000 calls) | 100 | 0.86 | 1.13 | 0.86 | 1.70 (secs for 300 calls) | 500 | 0.87 | 0.95 | 0.86 | 1.15 (secs for 4 calls) | . | Finding matrix inverse | ================================== | | contiguous | non-contiguous | ---------------------------------------------- | size | scipy | Numeric | scipy | Numeric | 20 | 0.76 | 1.24 | 0.76 | 1.35 (secs for 2000 calls) | 100 | 2.20 | 3.66 | 2.20 | 4.07 (secs for 300 calls) | 500 | 2.32 | 3.31 | 2.30 | 3.51 (secs for 4 calls) | . | ---------------------------------------------------------------------- | Ran 35 tests in 50.385s | | OK `---- These numbers are repeatably the same (+-2 in the 1/100 s). I don't know what the problem is, but it looks as scipy scales worse than Numeric? Interesting to note is also that here Numeric seems to be much faster, and scale much better, than for Travis -- who has the faster CPU (1.1GHz Athlon vs. 800MHz PIII)! This was run on a dual CPU PIII/800, RedHat-7.0 + gcc-3.0.4 + python-2.2.1c1. ATLAS was compiled with the system kgcc, which is egcs-1.1.2. scipy is compiled with gcc-3.0.4, compiler options ,---- | -march=i686 -O3 -unroll_loops -fPIC `---- The machine was busy with a single-thread (single cpu) job, but plenty of physical RAM was free (about half of 512 MB). Greetings, Jochen -- University of North Carolina phone: +1-919-962-4403 Department of Chemistry phone: +1-919-962-1579 Venable Hall CB#3290 (Kenan C148) fax: +1-919-843-6041 Chapel Hill, NC 27599, USA GnuPG key: 44BCCD8E

5 13

Finished linalg2->linalg transformation
by pearu＠scipy.org April 7, 2002

April 7, 2002

Hi, I have finished replacing linalg with linalg2. The old version of linalg is copied into directory linalg/linalg1. Later linalg1 will be removed from the CVS tree. I have tested that the new setup builds from tar source distribution on Woody Debian with Python 2.1.2, gcc-2.95.4, ATLAS-3.3.13. When updating SciPy from CVS, I suggest first removing linalg and linalg2 directories from your local CVS tree. Also, if you have installed scipy, it is a good idea to remove all traces of scipy from site-packages directory (but keep scipy_distutils, otherwise you have to reinstall f2py). Enjoy, Pearu

1 0

Warning: CVS is goint to be unstable due to linalg2 -> linalg
by pearu＠scipy.org April 7, 2002

April 7, 2002

... until further notice. Pearu

1 0

installation and linalg2 bench
by arnd.baecker＠physik.uni-ulm.de April 7, 2002

April 7, 2002

Hi, I successfully installed a CVS scipy (cvs_version = (1, 68, 1455, 3473)) - everything went quite smooth (I found site.cfg very convenient!). There were just 2 [[Installation was on a debian woody, self-compiled Python 2.2.1c2 (#1, Apr 4 2002, 18:54:47) [GCC 2.95.4 20011006 (Debian prerelease)] on linux2, self-compiled ATLAS 3.3.14, CVS Numeric 22.0a0 ]] Running scipy.test(10) gives the following messages, which I find a bit irritating (though they might not be crucial ?) 1.) creating test suite for: scipy.common !! FAILURE building test for scipy.common <string>:1: ImportError: No module named test_common (in ?) 2.) creating test suite for: scipy.stats.stats !! FAILURE building test for scipy.stats.stats <string>:1: ImportError: No module named test_stats (in ?) 3.) [...] 1st run(Numeric,compiled,speed up): 2.3194, 1.0310, 2.2498 2nd run(Numeric,compiled,speed up): 2.4009, 0.9876, 2.4310 .warning: specified build_dir '_bad_path_' does not exist or is or is not writable. Trying default locations ...warning: specified build_dir '..' does not exist or is or is not writable. Trying default locations .warning: specified build_dir '_bad_path_' does not exist or is or is not writable. Trying default locations ...warning: specified build_dir '..' does not exist or is or is not writable. Trying default locations .................................... test printing a value:2 ../home/abaecker/.python22_compiled/sc_9a25bc84add18fe6c75501f6b01bd84e1.cpp: In function `struct PyObject * compiled_func(PyObject *, PyObject *)': /home/abaecker/.python22_compiled/sc_9a25bc84add18fe6c75501f6b01bd84e1.cpp:418: no match for `Py::String & < int' /home/abaecker/PYTHON/lib/python2.2/site-packages/scipy/weave/CXX/Objects.hxx:390: candidates are: bool Py::Object::operator <(const Py::Object &) const /home/abaecker/PYTHON/lib/python2.2/site-packages/scipy/weave/CXX/Objects.hxx:1433: bool Py::operator <(const Py::SeqBase<Py::Object>::const_iterator &, const Py::SeqBase<Py::Object>::const_iterator &) /home/abaecker/PYTHON/lib/python2.2/site-packages/scipy/weave/CXX/Objects.hxx:1426: bool Py::operator <(const Py::SeqBase<Py::Object>::iterator &, const Py::SeqBase<Py::Object>::iterator &) ........ [...] ----------------------------------------------------------------------- Ran 304 tests in 859.017s 4.) Finally I did the benchmarking (note that this is on a PII, 350 MHz ... ;-) >>> import scipy.linalg2 >>> scipy.linalg2.basic.test() ................................ Finding matrix determinant ================================== | contiguous | non-contiguous ---------------------------------------------- size | scipy | Numeric | scipy | Numeric 20 | 1.14 | 2.47 | 1.13 | 2.84 (secs for 2000 calls) 100 | 2.21 | 8.24 | 2.18 | 9.58 (secs for 300 calls) 500 | 2.06 | 11.81 | 2.05 | 12.34 (secs for 4 calls) . Solving system of linear equations ================================== | contiguous | non-contiguous ---------------------------------------------- size | scipy | Numeric | scipy | Numeric 20 | 1.70 | 2.09 | 1.67 | 2.47 (secs for 2000 calls) 100 | 2.21 | 7.67 | 2.23 | 9.19 (secs for 300 calls) 500 | 2.01 | 11.55 | 2.03 | 12.24 (secs for 4 calls) . Finding matrix inverse ================================== | contiguous | non-contiguous ---------------------------------------------- size | scipy | Numeric | scipy | Numeric 20 | 2.22 | 4.27 | 2.25 | 4.53 (secs for 2000 calls) 100 | 5.78 | 23.75 | 5.77 | 24.99 (secs for 300 calls) 500 | 5.84 | 46.16 | 5.85 | 46.57 (secs for 4 calls) . ---------------------------------------------------------------------- Ran 35 tests in 294.343s Best, Arnd

1 0

cvs probs
by Jochen Küpper April 5, 2002

April 5, 2002

Ok, i try to run some more benchmarks. I do a cvs up -A, build, I get a ,---- | ... | scipy_base.fastumath fastumath needs fortran libraries 0 0 | building 'scipy_base.fastumath' extension | error: file '/home/jochen/source/numeric/scipy/scipy_base/scipy_base.fastumathmodule.c' does not exist `---- ^^^^^^^^^^^^^^^^^^^^^^ So I check what has changed since yesterday... ,---- | > cvs diff -u -D yesterday | ? linalg2/cblas2.pyf | ? linalg2/clapack2.pyf | ? linalg2/fblas2.pyf | ? linalg2/flapack2.pyf | ? scipy_base/mconf_lite.h | cvs server: Help.py no longer exists, no comparison available | cvs server: MANIFEST no longer exists, no comparison available | Index: __cvs_version__.py | =================================================================== | RCS file: /home/cvsroot/world/scipy/__cvs_version__.py,v | retrieving revision 1.68.1455.3520 | retrieving revision 1.68.1455.3499 | cvs [server aborted]: could not find desired version 1.68.1455.3499 in /home/cvsroot/world/scipy/__cvs_version__.py,v `---- What's going wrong here? Greetings, Jochen -- University of North Carolina phone: +1-919-962-4403 Department of Chemistry phone: +1-919-962-1579 Venable Hall CB#3290 (Kenan C148) fax: +1-919-843-6041 Chapel Hill, NC 27599, USA GnuPG key: 44BCCD8E

2 1

issues with distributing Numeric separately on windows
by eric April 5, 2002

April 5, 2002

Group, When upgrading to Python2.2 I re-discovered a windows specific issue that Travis O. and I fought with before releasing SciPy-0.1. It has to do with Numeric and fastumath. These guys work very closely together. On windows, the pre-compiled binaries available from source-forge are compiled with MSVC compiler. SciPy, and hence fastumath, is compiled with gcc. This works fine except for calls from Numeric C extensions into the C code of fastumath that return complex numbers. There is a binary incompatibility between MSVC and gcc in their structure layout that results in segmentation faults on complex number operations. So: >>> from scipy_base import * >>> array(1+1j)/1. will cause a seg fault if you use the standard Numeric module with SciPy. Some people balked when SciPy was distributed with Numeric last time. We've endeavored to separate things out this time so that that wasn't necessary. However, this is pretty much a show stopper on windows -- and was major reason we bundled them last time. I think we'll have to distribute a gcc compiled version of Numeric with SciPy on windows. This will get installed over (will clobber) your old installation of Numeric in the process. It does not affect Numeric's other than making it version 21.0. By this I mean, the gcc and MSVC versions will work identically and work with all your extension modules. If anyone has run into the binary incompatibility issue before and discovered a solution, I'd love to here it. I found the --native-struct flag for mingw, but it didn't seem to help. One other solution is to get SciPy working with MSVC. I think it might be possible (though there are some big issues), but it isn't a high priority. So, let me know if you have a solution, eric -- Eric Jones <eric at enthought.com> Enthought, Inc. [www.enthought.com and www.scipy.org] (512) 536-1057

4 3

CVS updates complete
by eric April 5, 2002

April 5, 2002

Well, not complete. There is still some cleanup/rearranging to do, but everything builds again. Also, all tests (304 of them) scipy.test(level=10) pass for me on Windows with Python 2.1.1 and Numeric 21.b1. A few notes. * scipy_base is now split out. It is meant to be a (somewhat) minimal function set of things often used in linalg, optimize, integrate, and other low level functions. It also holds any changes to the standard Numeric module. The only extension module at the moment is fastumath. * scipy_base has nearly 100% testing coverage (mostly thanks to Travis Oliphant). I think we need a few more tests, but it is very close to being a (reasonably) complete set of tests. * The issue with Numeric.Character with NumPy pre 21.x should be fixed. * We're temporarily using some python based ieee floating point code by Tim Peters while a few of the cephes functions (isnan and isfinite) get moved into fastumath. Please tests and let us know of failures. Also, the contents of scipy_base and any desired additions or omissions from it are open for comment. thanks, eric -- Eric Jones <eric at enthought.com> Enthought, Inc. [www.enthought.com and www.scipy.org] (512) 536-1057

6 22

Testing scipy_base
by pearu＠scipy.org April 4, 2002

April 4, 2002

Hi, While running scipy_base tests, I get ====================================================================== ERROR: check_complex1 (test_type_check.test_isnan) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.1/site-packages/scipy_base/tests/test_type_check.py", line 92, in check_complex1 assert(isnan(array(0+0j)/0.) == 1) ValueError: math domain error ====================================================================== ERROR: check_ind (test_type_check.test_isnan) ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/local/lib/python2.1/site-packages/scipy_base/tests/test_type_check.py", line 84, in check_ind assert(isnan(array((0.,))/0.) == 1) OverflowError: math range error <snip> I am using Python 2.1.2 (#1, Apr 1 2002, 18:23:14) [GCC 2.95.4 20011002 (Debian prerelease)] on linux2 and >>> Numeric.__version__ '20.2.1' Do you get these errors with the latest Numeric 21.0? On Windows? Any other ideas? Pearu

2 1

Re: [SciPy-dev] problems to find ATLAS during build
by Heiko Henkelmann April 3, 2002

April 3, 2002

And the content and location of my atlas directory: E:\USR\LIB\ATLAS MinGW>ls -l total 10308 -rw-r--r-- 1 henkelma 544 4587352 Mar 26 21:03 LIBATLAS.A -rw-r--r-- 1 henkelma 544 242394 Mar 26 20:43 LIBCBLAS.A -rw-r--r-- 1 henkelma 544 257842 Mar 26 21:06 libf77blas.a -rw-r--r-- 1 henkelma 544 5466190 Mar 27 20:01 liblapack.a ----- Original Message ----- From: "Heiko Henkelmann" <heiko(a)hhenkelmann.de> To: <scipy-dev(a)scipy.org> Sent: Wednesday, April 03, 2002 11:32 PM Subject: Re: [SciPy-dev] problems to find ATLAS during build > Sorry, I forgot to mention that I'm using Python 2.1 on a Windows box. > > > ----- Original Message ----- > From: "Heiko Henkelmann" <heiko(a)hhenkelmann.de> > To: <scipy-dev(a)scipy.org> > Sent: Wednesday, April 03, 2002 11:13 PM > Subject: [SciPy-dev] problems to find ATLAS during build > > > > > > Hello there, > > > > the following used to work before (last time with an update from Monday): > > > > > > file: build\generated_pyfs\fblas.pyf > > file: build\generated_pyfs\cblas.pyf > > atlas_info: > > NOT AVAILABLE > > > > Traceback (most recent call last): > > File "setup.py", line 127, in ? > > install_package() > > File "setup.py", line 94, in install_package > > config.extend([get_package_config(x,parent_package)for x in > > standard_package > > s]) > > File "setup.py", line 46, in get_package_config > > config = mod.configuration(parent) > > File "linalg2\setup_linalg2.py", line 29, in configuration > > raise AtlasNotFoundError,AtlasNotFoundError.__doc__ > > scipy_distutils.system_info.AtlasNotFoundError: > > Atlas (http://math-atlas.sourceforge.net/) libraries not found. > > Either install them in /usr/local/lib/atlas or /usr/lib/atlas > > and retry setup.py. One can use also ATLAS environment variable > > to indicate the location of Atlas libraries. > > > > > > Thanx > > > > Heiko > > > > > > _______________________________________________ > > Scipy-dev mailing list > > Scipy-dev(a)scipy.net > > http://www.scipy.net/mailman/listinfo/scipy-dev > > >

3 3

major changes on CVS...
by eric April 2, 2002

April 2, 2002

scipy_base just went through a major change. I've checked it in so that Travis O. can begin to clean some thing after the changes. The CVS will be broken for builds for a little while until we clean things up. I'd suggest not updating until the all clear signal is given if you rely on scipy for real work. I'll email when all seems to work. eric -- Eric Jones <eric at enthought.com> Enthought, Inc. [www.enthought.com and www.scipy.org] (512) 536-1057

1 0