Mailman 3 May 2002 - SciPy-User

ImportError: No module named fastumath
by Carl Staelin Aug. 15, 2002

Aug. 15, 2002

I grabbed scipy from CVS and built it last night, and I get an error when I try to use it: ImportError: No module named fastumath Some details on my system: OS: Linux, Mandrake 8.1 Python: python-2.1 Numerical: latest from CVS as of last night, 22.0a fypy2e: latest from CVS as of last night, 2.1.13.175.1250 ATLAS: 3.3.14 LAPACK: 3.0 When I look in /usr/lib/python2.1/site-packages/scipy_base, there is a file fastumath.so, so I am puzzled as to why the statement 'import scipy_base.fastumath' fails in scipy_base/__init__.py. Any suggestions welcome... Thanks, Carl Here is the debugging output: [staelin@firewall scipy]$ python -v # /usr/lib/python2.1/site.pyc matches /usr/lib/python2.1/site.py import site # precompiled from /usr/lib/python2.1/site.pyc # /usr/lib/python2.1/os.pyc matches /usr/lib/python2.1/os.py import os # precompiled from /usr/lib/python2.1/os.pyc import posix # builtin # /usr/lib/python2.1/posixpath.pyc matches /usr/lib/python2.1/posixpath.py import posixpath # precompiled from /usr/lib/python2.1/posixpath.pyc # /usr/lib/python2.1/stat.pyc matches /usr/lib/python2.1/stat.py import stat # precompiled from /usr/lib/python2.1/stat.pyc # /usr/lib/python2.1/UserDict.pyc matches /usr/lib/python2.1/UserDict.py import UserDict # precompiled from /usr/lib/python2.1/UserDict.pyc Python 2.1.1 (#1, Aug 30 2001, 17:36:05) [GCC 2.96 20000731 (Mandrake Linux 8.1 2.96-0.61mdk)] on linux-i386 Type "copyright", "credits" or "license" for more information. import readline # dynamically loaded from /usr/lib/python2.1/lib-dynload/readline.so >>> import scipy import scipy # directory /usr/lib/python2.1/site-packages/scipy # /usr/lib/python2.1/site-packages/scipy/__init__.pyc matches /usr/lib/python2.1/site-packages/scipy/__init__.py import scipy # precompiled from /usr/lib/python2.1/site-packages/scipy/__init__.pyc # /usr/lib/python2.1/site-packages/scipy/scipy_version.pyc matches /usr/lib/python2.1/site-packages/scipy/scipy_version.py import scipy.scipy_version # precompiled from /usr/lib/python2.1/site-packages/scipy/scipy_version.pyc # /usr/lib/python2.1/site-packages/scipy/__cvs_version__.pyc matches /usr/lib/python2.1/site-packages/scipy/__cvs_version__.py import scipy.__cvs_version__ # precompiled from /usr/lib/python2.1/site-packages/scipy/__cvs_version__.pyc # /usr/lib/python2.1/site-packages/Numeric/Numeric.pyc matches /usr/lib/python2.1/site-packages/Numeric/Numeric.py import Numeric # precompiled from /usr/lib/python2.1/site-packages/Numeric/Numeric.pyc # /usr/lib/python2.1/site-packages/Numeric/numeric_version.pyc matches /usr/lib/python2.1/site-packages/Numeric/numeric_version.py import numeric_version # precompiled from /usr/lib/python2.1/site-packages/Numeric/numeric_version.pyc import _numpy # dynamically loaded from /usr/lib/python2.1/site-packages/Numeric/_numpy.so import multiarray # dynamically loaded from /usr/lib/python2.1/site-packages/Numeric/multiarray.so import umath # dynamically loaded from /usr/lib/python2.1/site-packages/Numeric/umath.so # /usr/lib/python2.1/site-packages/Numeric/Precision.pyc matches /usr/lib/python2.1/site-packages/Numeric/Precision.py import Precision # precompiled from /usr/lib/python2.1/site-packages/Numeric/Precision.pyc # /usr/lib/python2.1/string.pyc matches /usr/lib/python2.1/string.py import string # precompiled from /usr/lib/python2.1/string.pyc import strop # dynamically loaded from /usr/lib/python2.1/lib-dynload/strop.so # /usr/lib/python2.1/types.pyc matches /usr/lib/python2.1/types.py import types # precompiled from /usr/lib/python2.1/types.pyc import math # dynamically loaded from /usr/lib/python2.1/lib-dynload/math.so # /usr/lib/python2.1/site-packages/Numeric/ArrayPrinter.pyc matches /usr/lib/python2.1/site-packages/Numeric/ArrayPrinter.py import ArrayPrinter # precompiled from /usr/lib/python2.1/site-packages/Numeric/ArrayPrinter.pyc # /usr/lib/python2.1/pickle.pyc has bad mtime import pickle # from /usr/lib/python2.1/pickle.py # can't create /usr/lib/python2.1/pickle.pyc # /usr/lib/python2.1/copy_reg.pyc matches /usr/lib/python2.1/copy_reg.py import copy_reg # precompiled from /usr/lib/python2.1/copy_reg.pyc import marshal # builtin import struct # dynamically loaded from /usr/lib/python2.1/lib-dynload/struct.so# /usr/lib/python2.1/re.pyc matches /usr/lib/python2.1/re.py import re # precompiled from /usr/lib/python2.1/re.pyc # /usr/lib/python2.1/sre.pyc matches /usr/lib/python2.1/sre.py import sre # precompiled from /usr/lib/python2.1/sre.pyc # /usr/lib/python2.1/sre_compile.pyc matches /usr/lib/python2.1/sre_compile.py import sre_compile # precompiled from /usr/lib/python2.1/sre_compile.pyc import _sre # builtin # /usr/lib/python2.1/sre_constants.pyc matches /usr/lib/python2.1/sre_constants.py import sre_constants # precompiled from /usr/lib/python2.1/sre_constants.pyc # /usr/lib/python2.1/sre_parse.pyc matches /usr/lib/python2.1/sre_parse.py import sre_parse # precompiled from /usr/lib/python2.1/sre_parse.pyc # /usr/lib/python2.1/StringIO.pyc has bad mtime import StringIO # from /usr/lib/python2.1/StringIO.py # can't create /usr/lib/python2.1/StringIO.pyc import errno # dynamically loaded from /usr/lib/python2.1/lib-dynload/errno.so # /usr/lib/python2.1/copy.pyc matches /usr/lib/python2.1/copy.py import copy # precompiled from /usr/lib/python2.1/copy.pyc import scipy_base # directory scipy_base # scipy_base/__init__.pyc matches scipy_base/__init__.py import scipy_base # precompiled from scipy_base/__init__.pyc Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib/python2.1/site-packages/scipy/__init__.py", line 29, in ? from scipy_base import * File "scipy_base/__init__.py", line 4, in ? import scipy_base.fastumath ImportError: No module named fastumath >>>

5 5

MDaemon Warning - Virus Found
by postmaster＠asiab2bonline.com May 29, 2002

May 29, 2002

The following message had attachment(s) which contained viruses: >From : scipy-user(a)scipy.net To : loryxu(a)263.net Subject : Japanese girl VS playboy Date : Message-ID: Attachment Virus name Action taken ------------------------------------------------------------------------------ cf1949426207.att Exploit.IFrame.FileDownloadRemoved Hro.bat I-Worm.Klez.h Removed

1 0

testing SciPy without ATLAS ;-(
by gvermeul＠polycnrs-gre.fr May 29, 2002

May 29, 2002

Sorry, Pearu, it's me again: After installing SciPy on a machine without ATLAS, scipy.test() resulted in ====================================================================== ERROR: Test persisting a function in the default catalog ---------------------------------------------------------------------- Traceback (most recent call last): File "/usr/lib/python2.1/site-packages/scipy/weave/tests/test_catalog.py", line 274, in check_add_function_persistent1 q.add_function_persistent('code',i) File "/usr/lib/python2.1/site-packages/scipy/weave/catalog.py", line 615, in add_function_persistent module = getmodule(function) File "/usr/lib/python2.1/site-packages/scipy/weave/catalog.py", line 66, in getmodule if mod and object in mod.__dict__.values(): File "/usr/lib/python2.1/site-packages/scipy_distutils/misc_util.py", line 13, in __getattr__ raise self._info[0],self._info[1] ImportError: /usr/lib/python2.1/site-packages/scipy/linalg/fblas.so: undefined symbol: srotmg_ ---------------------------------------------------------------------- Ran 447 tests in 2.017s FAILED (errors=2) <unittest.TextTestRunner instance at 0x853506c> >>> grepping the output of nm, I found "srotmg" back in the ATLAS libraries on another machine but not in the standard LAPACK/BLAS libraries. Gerard --------------------------------------------- This message was sent using Endymion MailMan. http://www.endymion.com/products/mailman/

4 9

Re: SEGFAULT from importing _minpack.so (was Re: [SciPy-user] install
by gvermeul＠polycnrs-gre.fr May 29, 2002

May 29, 2002

> > On Thu, 23 May 2002, Gerard Vermeulen wrote: > > > > > So, it looks like if there is something wrong with > > > > calling Fortran from C in this module. > > > > > > > > If there is nothing wrong with the compiler/linker flags > > > > produced by setup.py, there is something wrong with > > > > Mandrake's gcc/g77-2.96. > > > > > > If you do > > > > > > f2py -c -m fun minpack/chkder.f minpack/dpmpar.f > > > python -c 'import fun' > > > > > > what happens? SEGFAULT? Nothing? > > > > > SEGFAULT > > This is very interesting. If you can succesfully import large number of > extension modules that wrap various Fortran functions but only this > function, chkder, is causing import failures, and when looking the chkder > code I cannot see anything special in it, then I would like to suggest it > is a compiler bug. But on the other hand, I am not an expert on compilers > and it could be easily something else, something so obvious that we > cannot see it. I am starting to be very curious what is going on here. > > So, if you are willing to continue this bug tracking (I can easily > understand if you are not, then may be updating a compiler would be a > way to proceed), then the next step would be playing around > with minpack/chkder.f. Again, I would suggest commenting out various part > of it (the code itself is rather small) and run > > f2py -c -m fun minpack/chkder.f minpack/dpmpar.f > python -c 'import fun' > > until you can import succesfully. > Tried to play with this stuff on Mandrake-8.1 and everything worked. Then, I discovered that Mandrake-8.2 g77 comes with two libraries: libg2c.a and libg2c-pic.a. Guess this might win a femtosecond or a nanobyte (my supposition is that there is a libg2c WITHOUT an a libg2c WITH position independent code -- but I should take a look in their gcc-2.96 RPM package to be sure -- and where do you report gcc-2.96 bugs?). So, I modified build_flib.py to replace the g77 library "g2c" by "g2c-pic" and now I can happily do "from scipy import *". But running "scipy.test()" dies with .zswap:n=3 ....... ** On entry to DGESDD parameter number 12 had an illegal value [packer@venus packer]$ This did not happen with the Red Hat compiled SciPy that I untarred (this SciPy is probably linked against a rather vanilla Atlas, isn't it?) Anyhow, I plan to reinstall a gcc/g77-2.95.3 to rebuild the ATLAS libraries and to get rid of the libg2c-pic.a idiosyncrasy and will hopefully solve the DGESDD problem at the same time :-) I will let you know. And Pearu, thanks a lot for your help and patience. Gerard. PS: how fundamental is Atlas to SciPy? Installing ATLAS is a big obstacle for most users. Could it be possible to make ATLAS optional? I have still a few old machines (150-300 MHz) and hate the prospect of installing Atlas on those. --------------------------------------------- This message was sent using Endymion MailMan. http://www.endymion.com/products/mailman/

7 14

SciPy and I do not like libg2c-pic.a in Mandrake's gcc-2.96 and 3.xx
by Gerard Vermeulen May 27, 2002

May 27, 2002

Dear Gwenole, I am using Python -- interfaced to Numerical libraries -- for computing. When installing SciPy - a suite of scientific extensions for Python, see www.scipy.org - I ran into a problem that I finally traced to the requirement of having to link in libg2c-pic.a for dynamically loadable libraries. See: http://www.scipy.org/site_content/mailman?fn=scipy-user/2002-May/000402.html and other messages in the same thread. The developers of SciPy see this as a "Mandrake idiosyncrasy", and I do agree. Looking in the SRPMS of the gcc compilers, I see that libg2c-pic.a is new in Mandrake-8.2 and that it exists in gcc-2.96 and 3.0.x. No doubt you have a good reason for introducing libg2c-pic.a, but it is an innovation that made me loose quite a lot of time. And I am hoping that you will undo it. If not, can you explain why you have introduced libg2c-pic.a and if you know of other distro's following the same policy (using an identical name), please tell. It will be great if SciPy builds out of the box on future versions of Mandrake! Best regards -- Gerard Vermeulen PS: Mandrake-8.2 is great -- keep up the good work!

3 2

Re: [SciPy-user] compiler & lapack issues
by gvermeul＠polycnrs-gre.fr May 26, 2002

May 26, 2002

Pearu wrote: > > Fix: replace the following line at the end of subroutine gesdd in file > scipy/linalg/src/calc_lwork.f > > max_lwork = MAXWRK > > with > > max_lwork = MAX(MINWRK,MAXWRK) > > With this fix both official and patched LAPACK should work so that no > additional special hooks will be needed. I hope that you can confirm this. > patched LAPACK works fine. Thanks + regards -- Gerard --------------------------------------------- This message was sent using Endymion MailMan. http://www.endymion.com/products/mailman/

1 0

Re: [SciPy-user] compiler & lapack issues
by gvermeul＠polycnrs-gre.fr May 26, 2002

May 26, 2002

Pearu wrote: > > > test_svd fails (I commented out almost everything except for this test, but > > impossible the get rid of the zswap:n=3" message). No failure if I also > > comment out test_svd. > > I think zswap:n=3 comes from test_fblas.py. > > > > > I will integrate the latest patches in my lapack. Maybe a patch has been > > superseded by another. > > test_svd fails, with newest patches applied. So, I tried to mask dgesdd, as you suggested. Problem is that dgessd is linked into _flinalg.so and flapack.so. On my system, the one in flapack.so is called during the test. Attached (hackish) patch replaces dgesdd.o in both *.so's. So, in principle, it is possible to repair "broken" LAPACK installations. Gerard --------------------------------------------- This message was sent using Endymion MailMan. http://www.endymion.com/products/mailman/

2 1

Re: [SciPy-user] compiler & lapack issues
by gvermeul＠polycnrs-gre.fr May 26, 2002

May 26, 2002

> > Attached (hackish) patch replaces dgesdd.o in both *.so's. > So, in principle, it is possible to repair "broken" LAPACK > installations. > > Gerard In human readable format: --- SciPy-0.2.0_alpha_101.3442/linalg/setup_linalg.py.gv Fri May 24 11:01:07 2002 +++ SciPy-0.2.0_alpha_101.3442/linalg/setup_linalg.py Sun May 26 15:51:58 2002 @@ -39,12 +39,19 @@ 'flapack':['generic_flapack.pyf'], 'clapack':['generic_clapack.pyf']} + flapack_mask = [] + for f in ['dgesdd.f' + ]: + flapack_mask.append(os.path.join(local_path,'src',f)) + for mod_name,sources in mod_sources.items(): sources = [os.path.join(local_path,s) for s in sources] mod_file = os.path.join(local_path,mod_name+'.pyf') if dep_util.newer_group(sources,mod_file): generate_interface(mod_name,sources[0],mod_file) sources = filter(lambda s:s[-4:]!='.pyf',sources) + if mod_name in ['flapack']: + sources += flapack_mask ext_args = {'name':dot_join(parent_package,'linalg',mod_name), 'sources':[mod_file]+sources} dict_append(ext_args,**atlas_info) @@ -56,6 +63,7 @@ for f in ['det.f','lu.f', #'wrappers.c','inv.f', ]: flinalg.append(os.path.join(local_path,'src',f)) + flinalg += flapack_mask ext_args = {'name':dot_join(parent_package,'linalg','_flinalg'), 'sources':flinalg} dict_append(ext_args,**atlas_info) --------------------------------------------- This message was sent using Endymion MailMan. http://www.endymion.com/products/mailman/

1 0

Re: [SciPy-user] compiler & lapack issues
by gvermeul＠polycnrs-gre.fr May 26, 2002

May 26, 2002

Pearu wrote: > > > I just downloaded LAPACK patches from > > > > > > ftp://netlib2.cs.utk.edu/lapack/patch/src/ > > > > > > and compiled linalg with it. And I don't get this error. I even put a > > > print statement into dgesdd.f to see that it was actually used (indeed, > > > it was but in different places, not after the "zswap:n=3" message). > > > Can you determine which test in linalg/tests/test_*.py actually fails? test_svd fails (I commented out almost everything except for this test, but impossible the get rid of the zswap:n=3" message). No failure if I also comment out test_svd. > > > Also, where can I get Mandrake lapack.src.rpm's to see what they have > > > done? I will integrate the latest patches in my lapack. Maybe a patch has been superseded by another. BTW, a small glitch in build_flib.py (it is fairly dead code): def find_lib_directories(self): lib_dir = [] match = r'Reading specs from (.*)/specs' # works I think only for unix... exit_status, out_text = run_command('g77 -v') I would replace ('g77 -v') with ('%s -v' % self.f77_compiler) Request: I would like to be able to do something like: python setup.py build fc-compiler=/usr/local/bin/g77 or python setup.py build --fc-compiler=g77-2.95.3 Actually, there is less need to tweak the C compiler than the Fortran compiler. Would it not be nice to specify a Fortran compiler when running Python's configure (--with-fcc=...)? Now I have the choice of hardcoding a specific g77 in SciPy's distutils scripts or of relying on dynamic links. Gerard --------------------------------------------- This message was sent using Endymion MailMan. http://www.endymion.com/products/mailman/

2 1

Re: [SciPy-user] compiler & lapack issues
by gvermeul＠polycnrs-gre.fr May 25, 2002

May 25, 2002

> > On Sat, 25 May 2002 gvermeul(a)polycnrs-gre.fr wrote: > > > The fact that running scipy.test() on my system dies with > > > > .zswap:n=3 > > ....... ** On entry to DGESDD parameter number 12 had an illegal value > > [packer@venus packer]$ > > > > is related to a LAPACK issue. > > > > Mandrake's LAPACK rpms come with a lot of patches that have been released > > after "Update, Version 3.0: May 31, 2000", see > > http://netlib2.cs.utk.edu/lapack/release_notes.html > > > > This includes new versions of ?gesdd.f with corrected work space > > calculations. Probably, the parameter checks in the corresponding > > *.pyf files are out of sync with those post-latest-Update release. > > I just downloaded LAPACK patches from > > ftp://netlib2.cs.utk.edu/lapack/patch/src/ > > and compiled linalg with it. And I don't get this error. I even put a > print statement into dgesdd.f to see that it was actually used (indeed, > it was but in different places, not after the "zswap:n=3" message). > Can you determine which test in linalg/tests/test_*.py actually fails? > Also, where can I get Mandrake lapack.src.rpm's to see what they have > done? > You can get a Mandrake src.rpm (allthough advertised to be for the future Mandrake-8.3/9.0, it is still the same rpm) ftp://ftp.free.fr/mirrors/ftp.mandrake-linux.com/Mandrake-devel/contrib/SRP… Rebuilding LAPACK with g77-2.95.3 and without patches made this message disappear. Now, I am rebuilding the stuff with patches to check what happens (uninstalled g77-2.96, lapack rpms, ... - steps towards a build environment enforcing g77-2.95.3 AND g++-2.96, but leaving the choice between gcc-2.95.3 or 2.96). I am a bit reluctant to undo this, but if you insist.... I'd rather prefer to help to get the LAPACK patches integrated than hunt for Mandrake's gcc-2.96/g2c-pic related problems. Gerard --------------------------------------------- This message was sent using Endymion MailMan. http://www.endymion.com/products/mailman/

2 1