Mailman 3 January 2004 - SciPy-Dev

Re: [vtkusers] c api and python and vtk - with SWIG
by Prabhu Ramachandran Jan. 28, 2004

Jan. 28, 2004

>>>>> "MA" == Mark Asbach <mark.asbach(a)post.rwth-aachen.de> writes: >> I have a set of C codes that manipulate a data structure, one >> component of which is an array of doubles. I've managed to >> wrap the C codes with SWIG so I have access to the structure >> with the typical SWIG accessor functions, >> i.e. Sp4Array_data_get() What do I have to do to get this data >> array into vtk? MA> The way I took for my code is to just write the conversion to MA> vtk data structures in C, wrap them with SWIG and mix that MA> with the vtk python stuff. There is a bug in the current SWIG MA> adaptor code of VTK though, and I don't know, if David MA> committed my fix to the repository already (David?). In general and especially when you have Numeric arrays, this is much easier to do if you use Weave. I completely forgot about it and never posted about it, sorry! My apologies also for the cross-posting. Here are some details. http://www.scipy.org/site_content/weave Weave is a neat tool that lets you write C++ code straight inside your Python code. The first time the Python script is run, the C++ code is compiled to a module, and stored in a special place (all done automatically). The next time the script runs, the script will use the compiled module. Weave generates all the standard boiler plate code to convert the objects, return values etc. I've not played with generic SWIG pointers, but it should be possible to use too. A long while ago I wrote a small component that makes it trivial to embed VTK C++ code inside a VTK-Python script. If you use weave from CVS this is available. I am not sure if the tarball at the Weave web site has this part and if it does not please ask on the scipy-dev list that I've CC'd for clarifications etc. I've also not tested this under MSVC. Only under Linux with g++. YMMV. Anyway, here is a trivial example that I used to test with. I convert a reasonably large Numeric array to a vtkFloatArray. # ------------ try: from scipy import weave except ImportError: import weave import vtk import sys import Numeric import time # change these to suit your needs. inc_dirs=['/cvs/VTK/','/cvs/VTK/Common'] lib_dirs=['/cvs/VTK/bin'] def simple_test(): a = vtk.vtkStructuredPoints() # Your C++ code code=""" printf("Inline a->ClassName() == %s\n", a->GetClassName()); printf("Inline a->GetReferenceCount() == %d\n", a->GetReferenceCount()); """ # Now execute this. weave.inline(code, ['a'], include_dirs=inc_dirs, library_dirs=lib_dirs) def test(): arr = Numeric.arange(0, 10, 0.0001) print "Number of elements in array = ", arr.shape[0] v_arr = vtk.vtkFloatArray() ts = time.time() for i in range(arr.shape[0]): v_arr.InsertNextValue(arr[i]) print "Time taken to do it in pure Python =", time.time() - ts v_arr = vtk.vtkFloatArray() # Your C++ code code = """ int size = Narr[0]; for (int i=0; i<size; ++i) v_arr->InsertNextValue(arr[i]); """ ts = time.time() # Execute it. weave.inline(code, ['arr', 'v_arr'], include_dirs=inc_dirs, library_dirs=lib_dirs) # Note, if you need special headers for compilation, add them via # an extra keyword arg., headers=["vtkFoo.h"] print "Time taken to do it using Weave =", time.time() - ts print "Checking data." for i in range(v_arr.GetNumberOfTuples()): val = (v_arr.GetValue(i) -arr[i] ) assert (val < 1e-6), "i = %d, val= %f"%(i, val) print "OK." if __name__ == "__main__": simple_test() test() # ------------ The example is certainly trivial, but shows you how it works. This is also old code and I am not sure how things have changed recently. For more details check the weave docs. On my machine I get these results: Inline a->ClassName() == vtkStructuredPoints Inline a->GetReferenceCount() == 2 Number of elements in array = 100000 Time taken to do it in pure Python = 0.800973892212 Time taken to do it using Weave = 0.0328440666199 Checking data. OK. Which is an ~25 fold speed increase. Its been a while since I used/wrote this so I'm most probably not the right person to ask questions. I just wanted to let you folks know of this option. You might be better served asking questions on scipy-dev. cheers, prabhu

1 0

Error on Linux Compile
by Travis Oliphant Jan. 20, 2004

Jan. 20, 2004

Pearu, I checked out a recent version of SciPy and installed on my Mandrake 9.2 Linux box (hard-drive choked) After a clean build and install (no fftw or djbfft) I am getting the following error. undefined symbol: fftr8_scale1024 in _fftpack.so I will try and track this down, but wondered if you recognized this call and why I suddenly wouldn't have it on my system. -Travis O.

4 3

specfun compile errors on windows
by Travis N. Vaught Jan. 20, 2004

Jan. 20, 2004

Greetings, I finally have taken the time to look at our nightly builds of scipy for win32 and noticed that specfun is not compiling--as some have reported here: http://www.scipy.org/site_content/mailman?fn=scipy-user/2004-January/002 457.html and a link to the threaded discussion is here: http://www.scipy.org/site_content/mailman?fn=scipy-user/2004-January/thr ead.html Specifically you get an 'internal compiler error' when you have -O3 set (using no optimization setting seems to work). I've tried it with gcc ~3.2 and it compiles fine but 3.2.3 and a 3.3.1 (that comes with cygwin) were not able to compile it. Since specfun.f is a rather large file it would be impossible for me to reduce this to a simple case for reporting to the g77 folks or finding a workaround. Does anyone (Travis O.?) want to take a stab at this? (It would stink to have to specify gcc 3.2 or below as a build requirement on windows.) Thanks, Travis BTW Here's what I get with gcc 3.3.1: ....<a bunch of warnings>... Lib\special\specfun\specfun.f: In subroutine `cgama': Lib\special\specfun\specfun.f:7786: warning: `y1' might be used uninitialized in this function Lib\special\specfun\specfun.f:7794: warning: `na' might be used uninitialized in this function Lib\special\specfun\specfun.f: In subroutine `aswfb': Lib\special\specfun\specfun.f:7871: warning: `sw' might be used uninitialized in this function Lib\special\specfun\specfun.f:7881: Internal compiler error in emit_swap_insn, a t reg-stack.c:990 Please submit a full bug report, with preprocessed source if appropriate. See <URL:http://www.mingw.org/bugs.shtml> for instructions. warning: build_py: package init file 'Lib\tests\__init__.py' not found (or not a regular file) warning: build_py: package init file 'c:\temp\2.1\scipy\Lib\cluster\tests\__init __.py' not found (or not a regular file) warning: build_py: package init file 'c:\temp\2.1\scipy\Lib\fftpack\tests\__init __.py' not found (or not a regular file) ...<more warnings>... ........................ Travis N. Vaught Enthought, Inc. (512)536-1057 http://www.enthought.com ........................

4 7

Strang error on cygwin
by Travis Oliphant Jan. 19, 2004

Jan. 19, 2004

I periodically get this strange error which is probably due to the way I am using cygwin. I'm wondering if someone with more experience on cygwin could help me. Say I'm using the cygwin bash shell and I set my Path to grab the enthought Python binary first but the cygwin copy of gcc PATH=/cygdrive/c/Python23:/usr/bin:/bin Now, when I run python setup.py build --compiler=mingw32 install I get the following error: building 'scipy.cluster._vq' extension cygiwn_output_dir: build\temp.win32-2.3\Release here build\temp.win32-2.3\Release\lib\cluster\src\vq_wrap.o ...: build\temp.win32-2.3\Release lib\cluster\src\vq_wrap.o C:\cygwin\bin\gcc.exe -O2 -w -Wstrict-prototypes -Ic:\Python23\include -Ic:\Pyth on23\PC -c Lib\cluster\src/vq_wrap.cpp -o build\temp.win32-2.3\Release\lib\clust er\src\vq_wrap.o In file included from c:/Python23/include/Python.h:75, from Lib/cluster/src/vq_wrap.cpp:176: c:/Python23/include/intobject.h:41: error: syntax error before `(' token In file included from c:/Python23/include/Python.h:77, from Lib/cluster/src/vq_wrap.cpp:176: c:/Python23/include/longobject.h:37: error: `__int64' was not declared in this scope c:/Python23/include/longobject.h:39: error: syntax error before `*' token c:/Python23/include/longobject.h:40: error: syntax error before `(' token c:/Python23/include/longobject.h:41: error: syntax error before `(' token error: command 'gcc' failed with exit status 1 I'm wondering if this is somehow due to path issues, but it seems tied up in distutils. Am I being foolish to try and use the Enthought Python2.3 windows binary with the cygwin gcc compiler (the -mno-cygwin option that distutils provides ensures that the dll won't link against cygwin).? Thanks for any insight. -Travis O.

1 0

[SciPy-dev] Windows build, possible problem with scipy_distutils MinGW support
by Brett G. Olivier Jan. 12, 2004

Jan. 12, 2004

Hi I have been trying to build scipy on win2k (Python 2.3, scipy cvs ver. '0.2.1_252.4501') using MinGW. One problem in the build process is a strange call to a program "g" when building the cluster module (the other is a problem building the specfun library), a possible solution can be found by making a change in scipy_distutils: mingw32_support.py As far as the cluster build problem is concerned: compiling scipy with "python setup.py build --compiler=mingw32" results in the following error: "g -shared build\temp.win32-2.3\Release\lib\cluster\src\vq_wrap.o -Lc:\python23\libs -Lc:\python23\PCBuild -Lbuild\temp.win32-2.3 -lpython23 -lrootfind -lc_misc-lcephes -o build\lib.win32-2.3\scipy\cluster\_vq.pyd error: command 'g' failed: No such file or directory" I have tracked down the probable source of this problem in mingw32_support.py: Line 96: self.compiler_cxx = 'g++' If I understand it correctly MinGW uses python distutils UnixCCompiler which in unixccompiler.py has: Line 174: if target_lang == "c++" and self.compiler_cxx: Line 175: linker[0] = self.compiler_cxx[0] I think this is the source of the phantom "g" call. In mingw32_support.py altering Line 96: Line 96: self.compiler_cxx = 'g++' Line 96: self.compiler_cxx = ['g++'] seems to sort this problem out. I'm not at all sure if this is a proper solution to this problem or not, any comments or questions are more than welcome. Best regards Brett -- Brett G. Olivier (bgoli at sun ac za) Biochemistry Dept, Stellenbosch University Tel: +27-21-8085871 Fax: +27-218085863

2 1

band matrices in scipy
by Arnd Baecker Jan. 5, 2004

Jan. 5, 2004

Hi, I am in the progress of wrapping the Lapack dsbev.f routine (for band-matrices). Thanks to the amazing f2py I got pretty far (actually in addition the good docs and Fernandos notes helped a lot!). Once this is finished I would be happy to see this included in scipy ;-). But before this could happen I have a couple of questions (sorry, the text below is long, but I wanted to be explicit ;-): - Is the following way of calling dsbev from python alright, (eg., I omitted ldab on the python side.) Would that parameter be useful to someone so that it is better to expose it (the full details of my few pyf lines are at the end of this mail)? In [7]: MODULENAME.dsbev? Type: fortran String Form: <fortran object at 0x4028ebf0> Namespace: Interactive Docstring: dsbev - Function signature: w,z,info = dsbev(ab,[compute_v,lower]) Required arguments: ab : input rank-2 array('d') with bounds (ldab,*) Optional arguments: compute_v := 1 input int lower := 0 input int Return objects: w : rank-1 array('d') with bounds (n) z : rank-2 array('d') with bounds (ldz,ldz) info : int - How can one supply better doc-strings to this routine? I.e., help(MODULENAME.dsbev) just gives "Help on fortran: <fortran object>" and scipy.info gives the same as ? from Ipython. I would like to add a more detailed description and an example. Is this somehow possible to do from within the .pyf wrapper? (I know that this is a pure f2py question, so maybe I should ask this question as well on the f2py mailing list...) - And one more technical aspect (before I forget about it): Is the wrapping done such that for _large_ matrices no unnecessary copies of arrays are made? - One thing which puzzled me a bit (presumably it is alright) concerns the return object z which contains the eigenvectors if compute_v=1. For compute_v=0 I changed ldz in the wrapper such that it is of length 1 (it is never referenced by dsbev.f). So if someone is just interested in eigenvalues and not eigenvectors a dummy variable for z has to be given. Is this alright (for a more "low" level routine like this)? In principle one could write a wrapper a la scipy.linalg.eig for the band matrices, but for my purposes I don't mind about the (sometimes needed) dummy variable. - Is the following correct? To get this into scipy.linalg.flapack the lines from MODULENAME.pyf have to be added to generic_flapack.pyf and <tchar=s,d> and <type_in_c>* and <type_in> have to be added at the corresponding place. And that's all? - Unittests: I think it would be good to have a few unit-tests for these routines. Presently I just created a sample 10x10 band matrix (see below), converted this to upper band storage and compared the eigenvalues/eigenvectors of scipy.linalg.eig with those of dsbev. In principle one should do this for both upper and lower triangle storage and for single and double. Would something like that be a reasonable unit-test for inclusion in scipy or is there some better approach (better test-matrices etc.)? - Further routines I am thinking of wrapping in this context a) for symmetric band matrices: dsbevd, dsbevx b) for Hermitian band matrices: zhbev, zhbevd, zhbevx and their single precision partners. Any opinions? Many thanks, Arnd ############################# And here the full example plus test: MODULENAME.pyf: # ------ snip here ------------------- ! -*- f90 -*- python module MODULENAME ! in interface ! in :MODULENAME !subroutine dsbev(ab,compute_v,lower) ! in :MODULENAME:dsbev.f subroutine dsbev(ab,compute_v,lower,n,ldab,kd,w,z,ldz,work,info) ! in :MODULENAME:dsbev.f callstatement (*f2py_func)((compute_v?"V":"N"),(lower?"L":"U"),&n,&kd,ab,&ldab,w,z,&ldz,work,&info) callprotoargument char*,char*,int*,int*,double*,int*,double*,double*,int*,double*,int* double precision dimension(ldab,*) :: ab integer optional,intent(in):: compute_v = 1 check(compute_v==1||compute_v==0) compute_v integer optional,intent(in),check(lower==0||lower==1) :: lower = 0 ! FIXME: does one really need the ldab as optional argument ? !integer optional,check(shape(ab,0)==ldab),depend(ab) :: ldab=shape(ab,0) integer intent(hide),depend(ab) :: ldab=shape(ab,0) integer intent(hide),depend(ab) :: n=shape(ab,1) integer intent(hide),depend(ab) :: kd=shape(ab,0)-1 double precision dimension(n),intent(out),depend(n) :: w double precision dimension(ldz,ldz),intent(out),depend(ldz) :: z ! For compute_v=1 z is used and contains the eigenvectors integer intent(hide),depend(n) :: ldz=compute_v?n:1 double precision dimension(ldz,ldz),depend(ldz) :: z double precision dimension(MAX(1,3*n-1)),intent(hide),depend(n) :: work integer intent(out)::info end subroutine dsbev end interface end python module MODULENAME ! This file was auto-generated with f2py (version:2.37.233-1545). ! See http://cens.ioc.ee/projects/f2py2e/ ! and then modified ... # ------ snip here ------------------- Then (dsbev.f from lapack is needed for this) f2py --debug-capi -c MODULENAME.pyf dsbev.f -llapack and a small test is ########### snip here ########### import MODULENAME from Numeric import * from scipy import diag,linalg N=10 fullmat=diag(-1.0*ones(N-1),-1)+diag(-1.0*ones(N-1),1)+diag(1.0*ones(N)) KD=2 # number of diagonals above the diagonal LDAB=KD+1 mat=zeros( (LDAB,N),typecode=Float) for i in arange(LDAB): # Transform to band matrix mat[LDAB-i-1,i:N]=diag(fullmat,i) w,evec,info=MODULENAME.dsbev(mat) w_lin,evec_lin=linalg.eig(fullmat) print sort(w)-sort(w_lin.real) print evec[:,0] print evec_lin[:,0] ########### snip here ###########

3 3

weave moved to scipy_core in CVS
by Pearu Peterson Jan. 4, 2004

Jan. 4, 2004

Hi, weave module is now moved to scipy_core. This simplifies building scipy/chaco modules that need weave (that might not be installed when first checking out scipy/chaco modules). So, when updating scipy or chaco_all or any other CVS module that contained weave then you'll probably see the following message: cvs server: cannot open directory /home/cvsroot/world/scipy/Lib/weave: \ No such file or directory In that case you should remove your local copy of weave (or if you have made changes to local weave then make a backup) and re-run 'cvs update' or 'cvs co weave'. Regards, Pearu

1 0