Mailman 3 October 2014 - SciPy-User

Re: [SciPy-User] On using scipy.ndimage.correlate to find common pattern in an image
by Sergio Rojas Oct. 4, 2014

Oct. 4, 2014

1 0

On using scipy.ndimage.correlate to find common pattern in an image
by Sergio Rojas Oct. 3, 2014

Oct. 3, 2014

2 1

AMPGO - Take 3
by Andrea Gavana Oct. 1, 2014

Oct. 1, 2014

So, after receiving a bunch of requests for the AMPGO global optimization code and the associated benchmarks, I decided to put what I have on the public domain. I don't have that much time to work on it myself *but* I would still appreciate bug reports and feature requests. The AMPGO algorithm code and the 184 ND test functions - plus 20 1D test functions - live here: https://code.google.com/p/ampgo/ The algorithm itself can be tuned to use various local optimizers from the SciPy stack - and also from the OpenOpt stack, although the overall performances of the OpenOpt algorithms ranges between poor and abysmal - with the exception of the BOBYQA algorithm, and I'd say it's high time whoever is developing SciPy should include it as numerical optimization option. The AMPGO code as it stands is *not* suitable for inclusion into SciPy: the docstrings are almost non-existent, the return values are probably not compatible with the SciPy standards and in any case I am using SciPy 0.12.b1 still. Some time ago a kind soul provided me with the analytic definition of the gradient of the Tunnelling function inside AMPGO but I never managed to code it. This will give a great boost to AMPGO for problems with readily available gradient information. My main page of results and conclusions about global optimization algorithms available in Python is here: http://infinity77.net/global_optimization/ All the conclusions and results and ranking criteria are based on the *number of function evaluations*: everything else- like CPU time and internal algorithm performances, has been scrapped in the most brutal and merciless way. Most of us deal with real life problems where a single function evaluation (a simulation) may take hours to complete, so a few milliseconds gain in the internal algorithm processing feels like a joke. That,of course, has not stopped any of the "big" Global Optimization Projects (see COCONUT and funny friends), nor any of the reported benchmarks for "famous" solvers (BARON, I'm looking at you) from using ridiculous CPU-time metrics. Comments and suggestions are most welcome - especially if they help improving the overall performance of the AMPGO code (Tunnelling function gradient is a good start) or fixing bugs in the test functions - I'm sure there are a few lurking by, it's not that easy to stay focused while coding so much numerical optimization bruhaha. -- Andrea. "Imagination Is The Only Weapon In The War Against Reality." http://www.infinity77.net # ------------------------------------------------------------- # def ask_mailing_list_support(email): if mention_platform_and_version() and include_sample_app(): send_message(email) else: install_malware() erase_hard_drives() # ------------------------------------------------------------- #

1 0

Does SciPy already supersed NumPy?
by Sergio Rojas Oct. 1, 2014

Oct. 1, 2014

6 6

Virtual camera
by Antonelli Maria Rosaria Oct. 1, 2014

Oct. 1, 2014

Hi, I would like to realise à virtual camera in order to simulate the behaviour of an existing camera . Do you have any suggestion or idea around that ? Is there anything already available ? Where should I start with ? Best regards, Rosa

2 1

Tests filling up RAM on Linux
by Daπid Oct. 1, 2014

Oct. 1, 2014

Hi, Running the scipy tests I get a sudden increase in memory usage, using more than the 16 GB or RAM I have, and thus freezing the computer for a few seconds twice, along with several test failures. It is reproducible every time. I have attached the output of the tests: $ nosetests scipy > test_stdout 2> test_stderr I have also monitored the input with vmstat: $ vmstat -n 1 > testing_vmstat The first freeze occurs between 130 and 160 s, and the second, shorter, occurs I believe at 350 s. Scipy was installed via pip on a Fedora 20 box and linked against ATLAS compiled from the official rpm. In [2]: scipy.__version__ Out[2]: '0.14.0' In [3]: scipy.show_config() atlas_threads_info: libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas'] library_dirs = ['/usr/lib64/atlas-sse3'] define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] language = f77 include_dirs = ['/usr/include'] blas_opt_info: libraries = ['ptf77blas', 'ptcblas', 'atlas'] library_dirs = ['/usr/lib64/atlas-sse3'] define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] language = c include_dirs = ['/usr/include'] atlas_blas_threads_info: libraries = ['ptf77blas', 'ptcblas', 'atlas'] library_dirs = ['/usr/lib64/atlas-sse3'] define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] language = c include_dirs = ['/usr/include'] openblas_info: NOT AVAILABLE lapack_opt_info: libraries = ['lapack', 'ptf77blas', 'ptcblas', 'atlas'] library_dirs = ['/usr/lib64/atlas-sse3'] define_macros = [('ATLAS_INFO', '"\\"3.8.4\\""')] language = f77 include_dirs = ['/usr/include'] openblas_lapack_info: NOT AVAILABLE lapack_mkl_info: NOT AVAILABLE blas_mkl_info: NOT AVAILABLE mkl_info: NOT AVAILABLE $ python Python 2.7.5 (default, Sep 25 2014, 13:57:38) [GCC 4.8.3 20140911 (Red Hat 4.8.3-7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. Fedora has updated Python today [1], and I don't remember this happening before; but I am not sure. My laptop has the same configuration, but with Scipy 0.15.0.dev-704b554 and the tests don't use more than a 4% of the memory. /David. [1] http://koji.fedoraproject.org/koji/buildinfo?buildID=580875

2 1

Fwd: scipy.cluster.spectral_clustering numerical issues
by antonio vergari Oct. 1, 2014

Oct. 1, 2014

Hi everyone, I'm trying to use the implementation of the spectral clustering algorithm found in the scipy.cluster package on a quite large data matrix (~16000 rows and 16 columns, binary data). I am also trying to give the algorithm an already computed affinity matrix representing a gaussian kernel (like in Andrew Ng's paper http://ai.stanford.edu/~ang/papers/nips01-spectral.pdf). I am not able to user the default eigensolver 'arpack' since I do not have enough memory and it starts constantly swapping (I have 8 Gb, but the matrix alone is almost 2Gb). Turning to the other alternative, 'lobpcg', I've found that errors arise. These are numerical errors spanning from a 'Not Xth minor semi definite' to the presence of Nans and 0s in the computation while I vary the value of the k clusters to find and the sigma value in the kernel (the variance). I am wondering if I lack some piece of theory and/or computing the similarity matrix in a wrong way. I do not appear to get these errors on smaller, randomly generate, binary matrices. Here is a printed matrix for sigma=3.0: [[ 1. 0.60653066 0.94595947 ..., 0.71653131 0.8007374 0.57375342] [ 0.60653066 1. 0.57375342 ..., 0.67780958 0.67780958 0.84648172] [ 0.94595947 0.57375342 1. ..., 0.75746513 0.75746513 0.60653066] ..., [ 0.71653131 0.67780958 0.75746513 ..., 1. 0.71653131 0.71653131] [ 0.8007374 0.67780958 0.75746513 ..., 0.71653131 1. 0.64118039] [ 0.57375342 0.84648172 0.60653066 ..., 0.71653131 0.64118039 1. ]] I am attaching also a piece of code and the data as a minimal working example. Thanks in advance

1 0

Minimizing Monte Carlo simulation function
by Jeremy Sanders Oct. 1, 2014

Oct. 1, 2014

I have a function which returns a value computed using a Monte Carlo simulation. I'd like to minimize this function. It's also hard to compute the gradient for this function as the parameters are converted to integers internally (basically they are converted to array indices). I'd have thought that simulated annealing might be the best way to minimize this function. However, this is now deprecated in scipy. The replacement, basinhopping, appears to use scipy.minimize internally, so I think this relies on a function where the gradient can be computed. Is there are real replacement for annealing in this scenario? basinhopping doesn't seem to work very well when I tried it. It seems that it is wrong to deprecate simulated annealing, as it is a widely understood algorithm. Doing some tests, the most robust way of finding the minimum I have found so far is to use the MCMC emcee module. Thanks Jeremy Sanders.

6 9