Mailman 3 October 2012 - SciPy-User

Avoiding inner for loops??
by Martin De Kauwe Dec. 22, 2012

Dec. 22, 2012

Hi, I need to avoid (at least) two inner for loops in what I am trying to do otherwise my processing takes forever. What is the best way to transfer what I am doing into a more "numpy way"? Essentially I am trying to call a model again for various different parameter combinations. The example is fictional, by the grid_size would ideally grow > 500 and by doing so the processing speed becomes very slow the way I have set things up.. thanks. example. import numpy as np def fake_model(data1, data2, p1, p2, p3): """ complete nonsense """ return data1 + data2 * p1 * p2 * p3 data1 = np.random.rand(10) # the size of this arrays varies might be 10 might be 15 etc data2 = np.random.rand(10) # the size of this arrays varies might be 10 might be 15 etc obs = np.random.rand(10) # the size of this arrays varies might be 10 might be 15 etc grid_size = 10 # Ideally this would be a large number param1 = np.linspace(5.0, 350, grid_size) param2 = np.linspace(5.0, 550, grid_size) param3 = np.linspace(1E-8, 10.5, grid_size) ss = np.zeros(0) for p1 in param1: for p2 in param2: for p3 in param3: ans = fake_model(data1, data2, p1, p2, p3) ss = np.append(ss, np.sum(obs - ans)**2) print np.sum(obs - ans)**2

6 12

Request for usage examples on scipy.stats.rv_continuous and scipy.ndimage.grey_dilate(structure)
by Christoph Deil Dec. 5, 2012

Dec. 5, 2012

Hi, I am having some trouble figuring out how to use the following two aspects of scipy: 1) How can I create a random distribution given by some python formula, e.g. pdf(x) = x**2 in the range x = 1 to 2. The example given in the rv_continuous docstring doesn't work for me: In [1]: import matplotlib.pyplot as plt In [2]: numargs = generic.numargs AttributeError: type object 'numpy.generic' has no attribute 'numargs' Is it also possible to use rv_continuous for multidimensional distributions, say pdf(x,y)=x*y in the range x = 1 to 2 and y = 0 to 3? Or for pdfs that are just given numerically on a grid (say as the result of a gaussian_kde() ), not by a python function? It would be nice if someone could add a few examples that show rv_continuous usage to the docstring or tutorial. 2) How to use the 'structure' argument of scipy.ndimage.grey_dilate() correctly? There are some explanations at http://docs.scipy.org/doc/scipy/reference/tutorial/ndimage.html but I think they are inconsistent with the implementation: -------------------------------------------------------------- In [98]: a = zeros((7,7)); a[3,3]=1; a Out[98]: array([[ 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 1., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0.]]) In [100]: s = generate_binary_structure(2,2); s Out[100]: array([[ True, True, True], [ True, True, True], [ True, True, True]], dtype=bool) In [101]: grey_dilation(a,size=(3,3)) Out[101]: array([[ 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 1., 1., 1., 0., 0.], [ 0., 0., 1., 1., 1., 0., 0.], [ 0., 0., 1., 1., 1., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0.]]) In [102]: grey_dilation(a,size=(3,3),structure=s) Out[102]: array([[ 1., 1., 1., 1., 1., 1., 1.], [ 1., 1., 1., 1., 1., 1., 1.], [ 1., 1., 2., 2., 2., 1., 1.], [ 1., 1., 2., 2., 2., 1., 1.], [ 1., 1., 2., 2., 2., 1., 1.], [ 1., 1., 1., 1., 1., 1., 1.], [ 1., 1., 1., 1., 1., 1., 1.]]) In [103]: grey_dilation(a,structure=s) --------------------------------------------------------------------------- TypeError Traceback (most recent call last) /Users/deil/<ipython console> in <module>() /opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/scipy/ndimage/morphology.pyc in grey_dilation(input, size, footprint, structure, output, mode, cval, origin) 354 sz = footprint.shape[ii] 355 else: --> 356 sz = size[ii] 357 if not sz & 1: 358 origin[ii] -= 1 TypeError: 'NoneType' object is unsubscriptable In [104]: grey_dilation(a,footprint=s) Out[104]: array([[ 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 1., 1., 1., 0., 0.], [ 0., 0., 1., 1., 1., 0., 0.], [ 0., 0., 1., 1., 1., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0.], [ 0., 0., 0., 0., 0., 0., 0.]]) -------------------------------------------------------------- Why doesn't grey_dilation(a,structure=s) work? Why doesn't grey_dilation(a,size=(3,3),structure=s) give the same result as grey_dilation(a,size=(3,3),footprint=s)? I am running on numpy version 1.4.0 and scipy version 0.7.1 Thanks! --Christoph

8 11

Problem with Installation of Scipy on Macbook
by manisha pujari Nov. 22, 2012

Nov. 22, 2012

Hello everyone, This is the first time I am using scipy and I am having much trouble to just install it on my macbook. I am using Python 2.6 and numpy 1.6.0. I downloaded scipy0.9.0 tar.gz file from the web link http://sourceforge.net/projects/scipy/files/scipy/ and tried to install it from the source. But on giving the command scipy-0.9.0$ python setup.py build it is always giving the following problem : Warning: No configuration returned, assuming unavailable.blas_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec', '-I/System/Library/Frameworks/vecLib.framework/Headers'] non-existing path in 'scipy/io': 'docs' lapack_opt_info: FOUND: extra_link_args = ['-Wl,-framework', '-Wl,Accelerate'] define_macros = [('NO_ATLAS_INFO', 3)] extra_compile_args = ['-faltivec'] umfpack_info: libraries umfpack not found in /System/Library/Frameworks/Python.framework/Versions/2.6/lib libraries umfpack not found in /usr/local/lib libraries umfpack not found in /usr/lib /Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/system_info.py:463: UserWarning: UMFPACK sparse solver (http://www.cise.ufl.edu/research/sparse/umfpack/) not found. Directories to search for the libraries can be specified in the numpy/distutils/site.cfg file (section [umfpack]) or by setting the UMFPACK environment variable. warnings.warn(self.notfounderror.__doc__) NOT AVAILABLE Traceback (most recent call last): File "setup.py", line 181, in <module> setup_package() File "setup.py", line 173, in setup_package configuration=configuration ) File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/core.py", line 152, in setup config = configuration() File "setup.py", line 122, in configuration config.add_subpackage('scipy') File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/misc_util.py", line 972, in add_subpackage caller_level = 2) File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/misc_util.py", line 941, in get_subpackage caller_level = caller_level + 1) File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/misc_util.py", line 878, in _get_configuration_from_setup_py config = setup_module.configuration(*args) File "scipy/setup.py", line 20, in configuration config.add_subpackage('special') File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/misc_util.py", line 972, in add_subpackage caller_level = 2) File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/misc_util.py", line 941, in get_subpackage caller_level = caller_level + 1) File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/misc_util.py", line 878, in _get_configuration_from_setup_py config = setup_module.configuration(*args) File "scipy/special/setup.py", line 54, in configuration extra_info=get_info("npymath") File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/misc_util.py", line 2184, in get_info pkg_info = get_pkg_info(pkgname, dirs) File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/misc_util.py", line 2136, in get_pkg_info return read_config(pkgname, dirs) File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/npy_pkg_config.py", line 390, in read_config v = _read_config_imp(pkg_to_filename(pkgname), dirs) File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/npy_pkg_config.py", line 326, in _read_config_imp meta, vars, sections, reqs = _read_config(filenames) File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/npy_pkg_config.py", line 309, in _read_config meta, vars, sections, reqs = parse_config(f, dirs) File "/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/distutils/npy_pkg_config.py", line 281, in parse_config raise PkgNotFound("Could not find file(s) %s" % str(filenames)) numpy.distutils.npy_pkg_config.PkgNotFound: Could not find file(s) ['/Library/Python/2.6/site-packages/numpy-1.6.0-py2.6-macosx-10.6-universal.egg/numpy/core/lib/npy-pkg-config/npymath.ini'] I am unable to understand where and what exactly the problem is. Can anyone please help me? I will be really thankful for a response. -- Regards, Manisha

4 3

Single or double precision?
by Sturla Molden Nov. 9, 2012

Nov. 9, 2012

Sorry this might be a bit off-topic. But I'll ask on this list anyway: Given that a signal is sampled at 16 bits resolution (at an ADC), will it always be sufficient to store a filtered version at single precision? I.e. a float32 has a 23 bit mantissa, so the truncation error should be tiny compared to the digitization error from the 16 bits ADC. Or am I thinking wrongly about this? Usually I don't care and just use double precision everywhere. But I will save gigabytes of store space by using single precision here. Sturla

5 8

overflow in .sum() of dtype bool sparse matrix
by Yaroslav Halchenko Nov. 3, 2012

Nov. 3, 2012

I wonder if that is somehow considered a feature and manual casting is generally advised in such cases: calling .sum on a bool matrix can easily lead to overflows causing bogus results (works fine on ndarrays): % git describe --tags v0.4.3-6232-g43c7982 % PYTHONPATH=$PWD ../demo-scipy-sparse-negativesoverflow.py summing 128 booleans in <type 'numpy.ndarray'> leads to answer [128] summing 128 booleans in <class 'scipy.sparse.csc.csc_matrix'> leads to answer [[-128]] % cat ../demo-scipy-sparse-negativesoverflow.py #!/usr/bin/python import numpy as np import scipy.sparse as sp test = np.random.rand(128, 1) test_m= sp.csc_matrix(test) for t in test, test_m: test_bool=t.astype('bool') sum = test_bool.sum(axis=0) print "summing %d booleans in %s leads to answer %s" \ % (t.shape[0], test_bool.__class__, sum) -- Yaroslav O. Halchenko Postdoctoral Fellow, Department of Psychological and Brain Sciences Dartmouth College, 419 Moore Hall, Hinman Box 6207, Hanover, NH 03755 Phone: +1 (603) 646-9834 Fax: +1 (603) 646-1419 WWW: http://www.linkedin.com/in/yarik

2 1

Inconsistent conventions in scipy.interpolate
by Armando Serrano Lombillo Oct. 31, 2012

Oct. 31, 2012

I've recently been catched by the fact that scipy.interpolate.interp2d(x, y, z) expects z.shape=(len(y), len(x)) while scipy.interpolate.RectBivariateSpline(x, y, z) expects z.shape=(len(x), len(y)). I find this inconsistency quite annoying and error prone, is there a reason for it? Armando

2 2

Warnings --- why do they occur and how can I stop them?
by Virgil Stokes Oct. 31, 2012

Oct. 31, 2012

I have the following installed: NumPy 1.6.1 SciPy 0.11.0 on a Windows Vista (32-bit) platform with Python 2.7 I get the following warnings: D:\python27\lib\site-packages\scipy\io\matlab\mio4.py:15: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility from mio_utils import squeeze_element, chars_to_strings D:\python27\lib\site-packages\scipy\io\matlab\mio4.py:15: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility from mio_utils import squeeze_element, chars_to_strings D:\python27\lib\site-packages\scipy\io\matlab\mio5.py:96: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility from mio5_utils import VarReader5 D:\python27\lib\site-packages\scipy\io\matlab\mio5.py:96: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility from mio5_utils import VarReader5 When the following statement is executed from scipy import io Why does this occur and what can be done to fix this problem?

3 5

I'm new..numpy/scipy installation problems plz help!
by Rob Oct. 31, 2012

Oct. 31, 2012

Hello, I'm trying to install matplotlib to do some basic plotting for a wxpython GUI. From what I've read I need to install scipy and numpy as well. I've already got python 2.7 32 bit up and running with wxpython and some other stuff so I want to add the plotting capablity to this. On a clean build I got numpy working by just downloading the prebuilt binaries for python 2.7 32 bit and installed it with the msi installer. This created a numpy folder in site packages and I was able to import it and start using it without any errors. I tried to do the same thing for sci py and no luck. I get an error in aptana studios/eclipse saying it can't find scipy. I've been trying to figure this out for a while now.... Are there any dependencies I need to install? Am I really required to get a compiler and compile all this for windows 7? It seems extremely difficult to get all this working and I'm out of stuff to google. What do I need to do in addition to running the scipy and numpy installers from source forge? I thought you could basically just extract them to site packages, import the modules and away you go but that hans't been the case for me so far. Platform Windows 7 32 bit python 2.7 Thanks, Rob

2 3

Very simple IIR filters?
by Sturla Molden Oct. 30, 2012

Oct. 30, 2012

I have noticed that scipy.signal lacks the simplest class of IIR filters (in fact the one I use most frequently). Specifically: - RC (single-pole) - Notch - Band-pass (inverted notch) - Anti-DC (zero at DC) These are of course very easy to construct and use with sp.signal.filter (and/or filtfilt). But I think it might be beneficial for some users to have them in scipy. Due to their size, they can be make a bit faster by running the loop in Cython instead of using sp.signal.filter (though I've never had use for this optimization). Would this be a useful contribution? Sturla

6 8

Share memory between python an C++
by FDM Oct. 30, 2012

Oct. 30, 2012

Hello list, I have a couple of functions in the form of shared C++ libraries, and want to use them from within python. Some of them involve big chunks of data which could be represented easily using numpy data types. Therefore, I am searching for a way to call the C++ function, pass a reference or pointer as argument, pointing to memory I have allocated in python, such that I can use the result of the function w/o copying. It should be possible to hide technicalities from a python user. I would apprechiate any hint. Best, Wolfgang

6 5