Mailman 3 December 2006 - SciPy-User

Re: [SciPy-user] GA framework with automatic adjustment of mutation/crossover parameters ?
by Marek Wojciechowski 19 Dec '06

19 Dec '06

On Mon, 18 Dec 2006 18:00:06 -0000, <scipy-user-request(a)scipy.org> wrote: > Marek Wojciechowski wrote: >> On Tue, 12 Dec 2006 12:35:39 -0000, <scipy-user-request(a)scipy.org> >> wrote: >> >>> There are some GA frameworks and recipes around for Python (that one of >>> scipy (scipy.ga?) disapeared?). Which can be recommended? >> >> There is pikaia optimizer in fortran: >> http://www.hao.ucar.edu/Public/models/pikaia/pikaia.html >> >> I created a python wrapper of this routine (with f2py) and this is >> included in ffnet distribution. >> Look at http://sourceforge.net/projects/ffnet >> After downloadind and installing ffnet you are granted with pikaia in >> python. Just write: >> >> from pikaia import pikaia >> >> Look at pikaia docstring for info how to use it. >> > thanks, is there some info on the method used there - especially if/how > it will auto-adjust the mutation/crossover rates? > Robert Yes, there is some info. And more in pikaia userguide downloadable from their homepage. -- Marek Wojciechowski

1 0

scipy.signal.convolve speed depending on order of parameters
by Gael Varoquaux 18 Dec '06

18 Dec '06

Hi, One of my colleagues is complaining that the speed of scipy.signal.convolve depends on the order of the arrays passed to the convolve call. He woul like some magic optimisation here. I told him this was the kind of optimisation left to the user but he still isn't happy, so I am propagating his remark here to have other people's opinions. Cheers, Gaël

1 0

GA framework with automatic adjustment of mutation/crossover parameters ?
by Marek Wojciechowski 18 Dec '06

18 Dec '06

On Tue, 12 Dec 2006 12:35:39 -0000, <scipy-user-request(a)scipy.org> wrote: > There are some GA frameworks and recipes around for Python (that one of > scipy (scipy.ga?) disapeared?). Which can be recommended? There is pikaia optimizer in fortran: http://www.hao.ucar.edu/Public/models/pikaia/pikaia.html I created a python wrapper of this routine (with f2py) and this is included in ffnet distribution. Look at http://sourceforge.net/projects/ffnet After downloadind and installing ffnet you are granted with pikaia in python. Just write: from pikaia import pikaia Look at pikaia docstring for info how to use it. Greetings -- Marek Wojciechowski

2 1

Using fftpack.diff
by Neilen Marais 18 Dec '06

18 Dec '06

Hi I'm trying to get the DFT of the derivative of a real valued periodic signal. If I have the time domain signal in ts (where one period is stored in ts), I do: fs = scipy.fftpack.fft(ts) dfs = scipy.fftpack.diff(fs) dts = scipy.fftpack.ifft(dfs) However dts seems to be almost purely imaginary whereas ts is purely real. I haven't done DFT related stuff for a while, am I misunderstanding completely? -- you know its kind of tragic we live in the new world but we've lost the magic -- Battery 9 (www.battery9.co.za)

2 1

error building scipy on linux with mkl
by WB 15 Dec '06

15 Dec '06

hi, i am attempting to build scipy on a linux machine with mkl 9.0 installed. when i do python setup.py install i get lots of warnings, and then build failure with the following output: begin error out>> In file included from Lib/fftpack/src/drfft.c:7: Lib/fftpack/src/fftpack.h:44:22: error: mkl_dfti.h: No such file or directory In file included from Lib/fftpack/src/drfft.c:7: Lib/fftpack/src/fftpack.h:44:22: error: mkl_dfti.h: No such file or directory error: Command "gcc -pthread -fno-strict-aliasing -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m32 -march=i386 -mtune=generic -fasynchronous-unwind-tables -D_GNU_SOURCE -fPIC -fPIC -DSCIPY_MKL_H -DSCIPY_MKL_H -I/usr/local/include -I/usr/include -Ibuild/src.linux- i686-2.4 -I/usr/lib/python2.4/site-packages/numpy/core/include -I/usr/include/python2.4 -c Lib/fftpack/src/drfft.c -o build/temp.linux- i686-2.4/Lib/fftpack/src/drfft.o" failed with exit status 1 <<end error out so i checked to see if i had the file it is complaining about: [bryan@localhost ~]$ locate mkl_dfti.h /opt/intel/mkl/9.0/include/mkl_dfti.h and checked to see if that directory is on my LD_LIBRARY_PATH: [bryan@localhost ~]$ echo $LD_LIBRARY_PATH /opt/intel/mkl/9.0/lib/32:/opt/intel/cc/9.1.045/lib:/opt/intel/mkl/9.0/include anyone know what i am supposed to do to get this to work? thanks, bryan

1 1

Universal binary SciPy on OS X
by Josh Marshall 15 Dec '06

15 Dec '06

I have been trying to build a universal SciPy that works, so I can include it in a redistributable application I build with py2app. The one at http://pythonmac.org/packages/py24-fat/index.html is *incomplete*, it contains some dynamic libraries which are ppc only, which is what also happens when I try to build a universal binary myself.Some libs build fine, but some are only a single arch (the native one). The results of for FILE in `find . | grep '\.so'` ; do lipo -info $FILE ; done are: ------------------------------------------------------------------------ ----------- Architectures in the fat file: ./scipy/cluster/_vq.so are: i386 ppc Non-fat file: ./scipy/fftpack/_fftpack.so is architecture: ppc Non-fat file: ./scipy/fftpack/convolve.so is architecture: ppc Non-fat file: ./scipy/integrate/_odepack.so is architecture: ppc Non-fat file: ./scipy/integrate/_quadpack.so is architecture: ppc Non-fat file: ./scipy/integrate/vode.so is architecture: ppc Non-fat file: ./scipy/interpolate/_fitpack.so is architecture: ppc Non-fat file: ./scipy/interpolate/dfitpack.so is architecture: ppc Architectures in the fat file: ./scipy/io/numpyio.so are: i386 ppc Architectures in the fat file: ./scipy/lib/blas/cblas.so are: i386 ppc Non-fat file: ./scipy/lib/blas/fblas.so is architecture: ppc Architectures in the fat file: ./scipy/lib/lapack/atlas_version.so are: i386 ppc Non-fat file: ./scipy/lib/lapack/calc_lwork.so is architecture: ppc Architectures in the fat file: ./scipy/lib/lapack/clapack.so are: i386 ppc Architectures in the fat file: ./scipy/lib/lapack/flapack.so are: i386 ppc Non-fat file: ./scipy/linalg/_flinalg.so is architecture: ppc Non-fat file: ./scipy/linalg/_iterative.so is architecture: ppc Architectures in the fat file: ./scipy/linalg/atlas_version.so are: i386 ppc Non-fat file: ./scipy/linalg/calc_lwork.so is architecture: ppc Architectures in the fat file: ./scipy/linalg/cblas.so are: i386 ppc Architectures in the fat file: ./scipy/linalg/clapack.so are: i386 ppc Non-fat file: ./scipy/linalg/fblas.so is architecture: ppc Non-fat file: ./scipy/linalg/flapack.so is architecture: ppc Architectures in the fat file: ./scipy/linsolve/_csuperlu.so are: i386 ppc Architectures in the fat file: ./scipy/linsolve/_dsuperlu.so are: i386 ppc Architectures in the fat file: ./scipy/linsolve/_ssuperlu.so are: i386 ppc Architectures in the fat file: ./scipy/linsolve/_zsuperlu.so are: i386 ppc Architectures in the fat file: ./scipy/ndimage/_nd_image.so are: i386 ppc Non-fat file: ./scipy/odr/__odrpack.so is architecture: ppc Non-fat file: ./scipy/optimize/_cobyla.so is architecture: ppc Non-fat file: ./scipy/optimize/_lbfgsb.so is architecture: ppc Non-fat file: ./scipy/optimize/_minpack.so is architecture: ppc Architectures in the fat file: ./scipy/optimize/_zeros.so are: i386 ppc Non-fat file: ./scipy/optimize/minpack2.so is architecture: ppc Architectures in the fat file: ./scipy/optimize/moduleTNC.so are: i386 ppc Architectures in the fat file: ./scipy/signal/sigtools.so are: i386 ppc Architectures in the fat file: ./scipy/signal/spline.so are: i386 ppc Non-fat file: ./scipy/sparse/sparsetools.so is architecture: ppc Non-fat file: ./scipy/special/_cephes.so is architecture: ppc Non-fat file: ./scipy/special/specfun.so is architecture: ppc Non-fat file: ./scipy/stats/futil.so is architecture: ppc Non-fat file: ./scipy/stats/mvn.so is architecture: ppc Non-fat file: ./scipy/stats/statlib.so is architecture: ppc Architectures in the fat file: ./scipy/stsci/convolve/_correlate.so are: i386 ppc Architectures in the fat file: ./scipy/stsci/image/_combine.so are: i386 ppc ------------------------------------------------------------------------ ----------- So I didn't pursue this any further for the time being. Since I am not an expert on the SciPy build process, I'm trying to work around this. Instead I took your (Chris') versions from http://trichech.us/? page_id=4 , both ppc and intel SuperPacks. I yanked the scipy dirs out of the packages, and called them scipy-ppc and scipy-i386. Then what I did was lipo the shared libs together, as described on Apple's site: http://developer.apple.com/opensource/buildingopensourceuniversal.html Like so: mv scipy-ppc scipy-universal cd scipy-universal for FILE in `find . | grep '\.so'` ; do file $FILE ; done # shows all ppc libs for LIB in `find . | grep '\.so'` ; do lipo -create $LIB ../scipy- i386/$LIB -output $LIB ; done for FILE in `find . | grep '\.so'` ; do file $FILE ; done # shows all universal libs, yay! cd .. sudo cp -r scipy-universal $PYTHON_SITEPACKAGES/scipy Now this works fine on my G4. I won't have a chance to test on an Intel Mac until later in the week, but this looks promising. Chris, since the package seem to be identical apart from the shared libs, you could probably do this on the SuperPack and only need to distribute one version. Could I also have some advice on how to deal with this sort of stuff. I imagine that much of it is tied up with not having an Apple-blessed Fortran compiler. (Fingers crossed for Leopard). Thanks for all your help everyone, Josh

1 0

strange error in integrate.romb
by Gael Varoquaux 14 Dec '06

14 Dec '06

Hi, I don't really understand how integrate.romb is supposed to work. First of all its docstring is not really clear: """ Uses Romberg integration to integrate y(x) using N samples along the given axis which are assumed equally spaced with distance dx. The number of samples must be 1 + a non-negative power of two: N=2**k + 1 """ Reading this it is not clear that y is supposed to be a vector, and not a function. Second if I run it on a really simple test case, I get an error, and it seems the error is due to a trivial bug in the code: In [13]: x = arange(0,2**10+1) In [14]: integrate.romb(x) --------------------------------------------------------------------------- exceptions.TypeError Traceback (most recent call last) /home/varoquau/<ipython console> /usr/lib/python2.4/site-packages/scipy/integrate/quadrature.py in romb(y, dx, axis, show) 256 for i in range(2,k+1): 257 start >>= 1 --> 258 slice_R = tupleset(slice_R, slice(start,stop,step)) 259 step >>= 1 260 R[(i,1)] = 0.5*(R[(i-1,1)] + h*add.reduce(y[slice_R],axis)) TypeError: tupleset() takes exactly 3 arguments (2 given) In [15]: And indeed if you check out the code for tupleset, you find this : def tupleset(t, i, value): l = list(t) l[i] = value return tuple(l) It looks to me there has been some refactoring and the integrate.romb has been left out. Cheers, Gaël

2 2

Best/Safest way to parse NumPy array in C extension?
by Lou Pecora 14 Dec '06

14 Dec '06

I've done this two ways. Which is better or safer? Say I'm passing a NumPy array (float) to a C extension. One way to "parse" it is (ignoring checks on return values, etc. for now), PyArrayObject *mat; PyArg_ParseTuple(args, "O!", &PyArray_Type, &mat); or is this better, PyObject *Pymat; PyArrayObject *mat; PyArg_ParseTuple(args, "O", &Pymat); mat=PyArray_ContiguousFromObject(objin,NPY_DOUBLE, 2,2); The latter appears to have the constraint that the array be contiguous. Or is that an illusion and it's saying it _expects_ a Python object that has contiguous data? I've done both. Pointing C arrays to the resulting PyArrays' data works fine, but I fear one way or the other might be courting disaster. BTW, I do other checks on dimension, type, etc., but I left those out here for clarity. (I hope.) -- Lou Pecora My views are my own. ____________________________________________________________________________________ Do you Yahoo!? Everyone is raving about the all-new Yahoo! Mail beta. http://new.mail.yahoo.com

5 11

Re: [SciPy-user] feed-forward neural network for python
by Marek Wojciechowski 13 Dec '06

13 Dec '06

On Tue, 12 Dec 2006 12:35:39 -0000, <scipy-user-request(a)scipy.org> wrote: Robert Kern wrote: > Marek Wojciechowski wrote: >> Another solution would be to try to include ffnet to scipy distribution. > We would be happy to accept it if you change the license to BSD instead > of GPL. This could be done, but ffnet depends on networkx also and they use LGPL. I suspect that for inclusion in scipy, this dependence should be removed. Am I right? -- Marek Wojciechowski

2 1

GA framework with automatic adjustment of mutation/crossover parameters ?
by Robert 12 Dec '06

12 Dec '06

I'm looking for an efficient optimizer on a noisy high-dimensional and costly function. My own GA hack seems to be somewhat stiff and I find me trying too much around with different cooling speeds and other non-systematic screwing ... There are some GA frameworks and recipes around for Python (that one of scipy (scipy.ga?) disapeared?). Which can be recommended? The searching in my use cases is mainly on fixed length parameter vectors (float and integer ranges). Just a few use cases on chromosome-bits and complex variable length structure building. My main concern is about a smart, general and robust population alteration - an automatic selection of mutation/crossover-random-step-ranges and mutation/crossover-ratio. And possibly a dynamic population/selection scheme also. The mutation function accepts a vector with step-width's 0..inf (default 1.0) for each parameter/gene. And the crossover function accepts a scalar 0 .. 1.0 controlling the extent of crossover. Thus like: def mutate(obj, astepstd=[1.0,1.0,1.0,...]): ... def crossover(obj, other, extent=0.5): ... The good optimizer alg probably should be smart enough to dynamically auto-adjust based the optimization history (independently of the task): * astepstd's * extent * crossover/mutate rate (* dynamic population size and selection-scheme) Any recommendations or hints? Robert

1 0