Mailman 3 April 2006 - SciPy-User

latex listings package for python code
by Ryan Krauss April 5, 2006

April 5, 2006

This is a little off topic and Fernado is the guy who would know, but he is sort of out email contact right now, but does anyone know how to alter line spacing within Python code snipets in Latex using the listings package? The documentation for the package seems not to talk about it. I thought I found something in Google, but it doesn't seem to work. It would be great if it worked with floating and non-floating snipets. Thanks, Ryan

1 0

Linear programming with scipy
by Armando Serrano Lombillo April 4, 2006

April 4, 2006

Hello, I'm looking for a tool to do linear programming with Python. Has this been included in SciPy? Armando Serrano

1 0

Error using 0.4.8 on code that worked on 0.3.2
by Antonio Arauzo Azofra April 4, 2006

April 4, 2006

Hello everybody, I am almost new to Scipy. I was using the following in my code on 0.3.2 and it worked: >>> mat=[[1,2,3],[4,5,6],[7,8,9]] >>> import scipy >>> scipy.cov(mat) Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/lib/python2.4/site-packages/numpy/lib/function_base.py", line 649, in cov if (m.shape[0] == 1): AttributeError: 'list' object has no attribute 'shape' Now it is necessary to use always an array: >>> amat = array(mat) Traceback (most recent call last): File "<stdin>", line 1, in ? NameError: name 'array' is not defined >>> amat = scipy.array(mat) >>> scipy.cov(amat) array([[ 9., 9., 9.], [ 9., 9., 9.], [ 9., 9., 9.]]) Is this better? If it is, (by no means I try to critize good work made on scipy) maybe it would be better if 'cov' funtion checked the type before begining, and throw an more meaningfull error. :-? Just a suggestion. Also I had problems because cov function returns an integer if matrix is 1x1. I have used this workarround but I am not sure if this is the best way to work with Scipy. Any suggestions? ----------------------------------------------------------- ... disp1 = scipy.cov(class1) #within class1 scatter matrix disp2 = scipy.cov(class2) #within class2 scatter matrix #We allways want covariance matrices even if they are 1 by 1 if not type(disp1) == type( scipy.array(0) ): disp1 = scipy.array([[disp1]]) if not type(disp2) == type( scipy.array(0) ): disp2 = scipy.array([[disp2]]) aux = scipy.add(disp1,disp2) aux = scipy.divide(aux,2) ... ----------------------------------------------------------- -- Regards, Antonio Arauzo Azofra

1 0

Band pass filter design
by vincenzo cacciatore April 4, 2006

April 4, 2006

Hi all, i would like to design a high pass filter with scipy.signal module. This is the code i'm using to: import scipy.signal as signal import scipy #first of all i design the lowpass fir filter. This is a 10 taps filter with cutoff frequency =1 (as help tell me to do) lpwindow=signal.firwin(10,1) #with the following instruction i'm creating a band pass filter from the low pass one bpwindow=signal.lp2bp(lpwindow,1,0.5,0.2) My problem is that the band-pass filter obtained with lp2bp function is 16 taps one! How is it possible?? thanks, Vincenzo

2 1

Statistics review months
by Robert Kern April 3, 2006

April 3, 2006

In the interest of improving the quality of the scipy.stats package, I hereby declare April and May of 2006 to be Statistics Review Months. I propose that we set ourselves a goal to review each function in stats.py and morestats.py (and a few others) for correctness and completeness of implementation by the end of May. By my count, that's about 2.5 functions every day. Surely this is a reasonable amount of effort for a rather large payoff: a robust, well-tested and thorough statistics library. I have added a Wiki page describing the details: http://projects.scipy.org/scipy/scipy/wiki/StatisticsReview Barring any objections, I will be irretrievably creating the ~150 tickets or so for all of the functions to be reviewed later tonight. So if you object, act fast! [Disclosure: this idea isn't mine. Eric Jones mentioned it to me once, and I'm just running with it.] -- Robert Kern robert.kern(a)gmail.com "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

8 7

stats review: std/var and samplestd/samplevar
by Zachary Pincus April 3, 2006

April 3, 2006

Hi folks - It appears to me that the scipy.stats implementations for calculating sample variances and population variances (and hence standard deviations too) are somehow reversed. Specifically, the variance of an entire population is calculated with a denominator of the population size N. The variance of a sample from a population is either estimated using a denominator of the sample size n (to obtain a biased estimate) or 1-n (to obtain an unbiased estimate). Note that saying "sample variance" does not imply the use of the 1-n estimator, as there are cases in which the biased estimator may legitimately be used.(*) see e.g.: http://en.wikipedia.org/wiki/Variance http://en.wikipedia.org/wiki/Standard_deviation However, scipy.stats.std and scipy.stats.var use 1-N, while scipy.stats.samplestd and scipy.stats.samplevar use N. This is clearly incorrect notation any way you slice it. I would propose to have: (1) scipy.stats.var and scipy.stats.std -- use N as the denominator (2) scipy.stats.samplevar and scipy.stats.samplesdt -- at least use n-1 as the denominator. Better would be to deprecate / remove them because as above "sample variance" is ambiguous. (3) scipy.stats.var_unbiased -- use n-1 as denominator. As per the note below, there is no general unbiased estimator of the standard deviation, and so there should be no scipy.stats.std_unbiased function. (See the wikipedia entry and also http://www.itl.nist.gov/ div898/handbook/pmc/section3/pmc32.htm ) I feel vaguely that the N-1 estimator is always problematic, because if you have a small enough sample that it makes a difference, you've got bigger problems than using N or N-1. Not that these problems are insurmountable, but you've got to have some statistical savvy to deal properly with them. As such, I think that the default functions (var and std) should just return the population statistics. But reasonable people may disagree. Zach Pincus Program in Biomedical Informatics and Department of Biochemistry Stanford University School of Medicine (*) E.g.: While it is possible to estimate the variance in an unbiased manner, estimating the standard deviation of a population from a sample without bias is actually impossible without assumptions about the population. (There is a complex correction factor for samples from normal populations discussed on the NIST page.) Moreover, though the (N-1)-denominated estimator of the variance is unbiased, the estimator itself has a greater variance around the true value than the N-denominated estimator. As such, using the unbiased estimator can sap statistical power from some tests. This is why sometimes one might use the N-denominated estimator for the sample variance.

3 3

Floating point exception in scipy
by Rob Hetland April 3, 2006

April 3, 2006

Compiled on an Intel Mac os X using gcc 4.0.1 (the only one available on intel macs) and gfortran (from hpc.sf.net). Python is MacPython Universal build 2.4.3. Numpy compiles without a hitch, and tests with no errors. SciPy also compiles without errors, but I get a floating point exception when trying to test scipy. This includes doing scipy.test (10,10). Below are the details of some attempts. I'm really not sure where to begin, as it compiles fine. Jordan Mantha has claimed success basically following the PPC build instructions with this compiler configuration, but I have not had any luck. I have also tried to exclude modules that need umfpack, but that also failed in a similar way. Any ideas? -Rob mire:~/src/python$ python Python 2.4.3 (#1, Mar 30 2006, 11:02:16) [GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import scipy import linsolve.umfpack -> failed: No module named _umfpack >>> scipy.test(10,10) Floating point exception mire:~/src/python$ python Python 2.4.3 (#1, Mar 30 2006, 11:02:16) [GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import scipy.special import linsolve.umfpack -> failed: No module named _umfpack Floating point exception mire:~/src/python$ python Python 2.4.3 (#1, Mar 30 2006, 11:02:16) [GCC 4.0.1 (Apple Computer, Inc. build 5250)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> from scipy import * import linsolve.umfpack -> failed: No module named _umfpack Floating point exception ----- Rob Hetland, Assistant Professor Dept of Oceanography, Texas A&M University p: 979-458-0096, f: 979-845-6331 e: hetland(a)tamu.edu, w: http://pong.tamu.edu

3 2

building numpy fails
by Steve Schmerler April 3, 2006

April 3, 2006

The latest numpy svn checkout fails to build (Python 2.3.5) when trying to call tempfile.mktemp(). The module index says: mktemp([suffix[, prefix[, dir]]]) Deprecated since release 2.3. Use mkstemp() instead. [...] ------------------------------------------------------------------------------------ [...] Traceback (most recent call last): File "setup.py", line 84, in ? setup_package() File "setup.py", line 77, in setup_package setup( configuration=configuration ) File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/core.py", line 152, in setup return old_setup(**new_attr) File "/usr/lib/python2.3/distutils/core.py", line 149, in setup dist.run_commands() File "/usr/lib/python2.3/distutils/dist.py", line 907, in run_commands self.run_command(cmd) File "/usr/lib/python2.3/distutils/dist.py", line 927, in run_command cmd_obj.run() File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/command/install.py", line 11, in run r = old_install.run(self) File "/usr/lib/python2.3/distutils/command/install.py", line 506, in run self.run_command('build') File "/usr/lib/python2.3/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/lib/python2.3/distutils/dist.py", line 927, in run_command cmd_obj.run() File "/usr/lib/python2.3/distutils/command/build.py", line 107, in run self.run_command(cmd_name) File "/usr/lib/python2.3/distutils/cmd.py", line 333, in run_command self.distribution.run_command(command) File "/usr/lib/python2.3/distutils/dist.py", line 927, in run_command cmd_obj.run() File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/command/build_src.py", line 84, in run self.build_sources() File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/command/build_src.py", line 103, in build_sources self.build_extension_sources(ext) File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/command/build_src.py", line 209, in build_extension_sources sources = self.generate_sources(sources, ext) File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/command/build_src.py", line 267, in generate_sources source = func(extension, build_dir) File "numpy/core/setup.py", line 35, in generate_config_h library_dirs = default_lib_dirs) File "/usr/lib/python2.3/distutils/command/config.py", line 278, in try_run self._check_compiler() File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/command/config.py", line 35, in _check_compiler self.fcompiler.customize(self.distribution) File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/fcompiler/__init__.py", line 294, in customize oflags = self.__get_flags(self.get_flags_opt,'FOPT',(conf,'opt')) File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/fcompiler/__init__.py", line 511, in __get_flags var = command() File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/fcompiler/gnu.py", line 122, in get_flags_opt if self.get_version()<='3.3.3': File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/ccompiler.py", line 251, in CCompiler_get_version status, output = exec_command(cmd,use_tee=0) File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/exec_command.py", line 254, in exec_command use_tee=use_tee, File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/exec_command.py", line 279, in _exec_command_posix tmpfile = tempfile.mktemp() AttributeError: 'module' object has no attribute 'mktemp' ------------------------------------------------------------------------------------ Replacing mktemp() by mkstemp() doesn't help: [...] File "/home/elcorto/install/python/scipy/svn/numpy/numpy/distutils/exec_command.py", line 279, in _exec_command_posix tmpfile = tempfile.mkstemp() AttributeError: 'module' object has no attribute 'mkstemp' but doing it in an interactive session works: In [1]: import tempfile In [2]: tempfile.mkstemp() Out[2]: (3, '/tmp/tmp0Iv8K7') What's going on? cheers, steve -- Random number generation is the art of producing pure gibberish as quickly as possible.

3 4

Filtering high frquency noise
by Yannick Dirou April 3, 2006

April 3, 2006

Hello, I have a multirate signal (from 18 to 24 sample per day), and if i plot it i see something like "high frequency" noise (actually not that high but higher than the remaining of the signal), i thought about using a median filter but this is not good for a multirate signal, then i though i could use scipy filters to do the job, unfortunately i know nearly nothing in filter design, and don't know how to do the job. Is there a tutorial or simple example to design a low pass filter? the signal data is made of a datetime in epoch format and the measured value. Thanks in advance, Yannick

4 3

Re: [SciPy-user] Statistics review months
by Pierre GM April 3, 2006

April 3, 2006

Robert, Excellent initative, thanks a lot ! Before getting too involved, I have a question: should the functions support MaskedArrays (when possible) ? I think about var/std (already available for MA in this patch, that should be checked before inclusion [http://projects.scipy.org/scipy/numpy/attachment/wiki/MaskedArray/ma-200603…]), or median, in particular...

1 0