Mailman 3 December 2006 - NumPy-Discussion

test issue
by belinda thom 29 Dec '06

29 Dec '06

Hello, I've been going thru Dave Kuhlman's "SciPy Course Outline" and found out about test functions -- very cool. Except that on my end, not all tests pass (appended below). Is this a problem for other people? Is it something I should worry about? Here's my setup: Mac G5 w/OS X 10.4.8, using MacPython 2.4, numpy.__version__ is 1.0, matplotlib.__version__ 0.87.7 and Numeric.__version__ 24.2 Thanks, --b ========== In [94]: import numpy In [95]: numpy.test() Found 13 tests for numpy.core.umath Found 9 tests for numpy.lib.arraysetops Found 3 tests for numpy.fft.helper Found 1 tests for numpy.lib.ufunclike Found 4 tests for numpy.ctypeslib Found 2 tests for numpy.lib.polynomial Found 8 tests for numpy.core.records Found 26 tests for numpy.core.numeric Found 5 tests for numpy.distutils.misc_util Found 3 tests for numpy.lib.getlimits Found 31 tests for numpy.core.numerictypes Found 4 tests for numpy.core.scalarmath Found 12 tests for numpy.lib.twodim_base Found 47 tests for numpy.lib.shape_base Found 4 tests for numpy.lib.index_tricks Found 32 tests for numpy.linalg.linalg Found 42 tests for numpy.lib.type_check Found 184 tests for numpy.core.multiarray Found 36 tests for numpy.core.ma Found 10 tests for numpy.core.defmatrix Found 41 tests for numpy.lib.function_base Found 0 tests for __main__ ........................................................................ ........................................................................ ........................................................................ ........................................................................ ........................................................................ F....................................................................... ........................................................................ ............. ====================================================================== FAIL: Ticket #112 ---------------------------------------------------------------------- Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.4/lib/ python2.4/site-packages/numpy/core/tests/test_regression.py", line 220, in check_longfloat_repr assert(str(a)[1:9] == str(a[0])[:8]) AssertionError ---------------------------------------------------------------------- Ran 517 tests in 1.241s FAILED (failures=1) <unittest.TextTestRunner object at 0x6194f30>

1 0

Newbie Question, Probability
by Mark Janikas 29 Dec '06

29 Dec '06

Hello all, Is there a way to get probability values for the various families of distributions in numpy? I.e. ala R: > pnorm(1.96, mean = 0 , sd = 1) [1] 0.9750021 # for the normal > pt(1.65, df=100) [1] 0.9489597 # for student t Any suggestions would be greatly appreciated. Mark Janikas Product Engineer ESRI, Geoprocessing 380 New York St. Redlands, CA 92373 909-793-2853 (2563) mjanikas(a)esri.com

13 29

optimize.fmin_cg bug
by Nadav Horesh 28 Dec '06

28 Dec '06

optimize.fmin_cg fails when the function to be minimized returns a numpy scalar: >>> from numpy import * >>> from scipy import optimize >>> f = lambda x: exp(x)-x >>> df = lambda x: exp(x)-1.0 >>> scipy.optimize.fmin(f, [0.2]) Traceback (most recent call last): File "<pyshell#4>", line 1, in <module> scipy.optimize.fmin(f, [0.2]) NameError: name 'scipy' is not defined >>> optimize.fmin(f, [0.2]) Optimization terminated successfully. Current function value: 1.000000 Iterations: 15 Function evaluations: 30 array([ 3.88578059e-16]) >>> optimize.fmin_cg(f, [0.2], df) Traceback (most recent call last): File "<pyshell#6>", line 1, in <module> optimize.fmin_cg(f, [0.2], df) File "C:\Python25\Lib\site-packages\scipy\optimize\optimize.py", line 855, in fmin_cg old_fval_backup,old_old_fval_backup) File "C:\Python25\Lib\site-packages\scipy\optimize\optimize.py", line 471, in line_search phi0, derphi0, c1, c2) File "C:\Python25\Lib\site-packages\scipy\optimize\optimize.py", line 359, in zoom a_j = _cubicmin(a_lo, phi_lo, derphi_lo, a_hi, phi_hi, a_rec, phi_rec) File "C:\Python25\Lib\site-packages\scipy\optimize\optimize.py", line 309, in _cubicmin [A,B] = numpy.dot([[dc**2, -db**2],[-dc**3, db**3]],[fb-fa-C*db,fc-fa-C*dc]) ValueError: objects are not aligned # Here is the cure: Make f return a python float >>> f = lambda x: float(exp(x)-x) >>> optimize.fmin_cg(f, [0.2], df) Optimization terminated successfully. Current function value: 1.000000 Iterations: 2 Function evaluations: 14 Gradient evaluations: 8 array([ -8.15285339e-14]) I had this error with scipy 0.5.2 and scipy from svn. Nadav.

1 0

extracting values from an array
by Eric Emsellem 28 Dec '06

28 Dec '06

Hi, I have a simple problem of extracting a subarray from a bigger one, for which I don't find an elegant/easy solution. I tried using searchsorted, or other tricks but I always end up with many "if" statements, testing all cases (also because searchsorted does not work on arrays which are sorted in decreasing order). There must be an elegant solution but I cannot find it. ==> I have one array of floats which has e.g. regular steps (as produced by arange) but can either be decreasing or increasing as for: x = arange(0.,1.,0.1) ## or x = arange(0.,-1.,-0.1) I also have another "data" y array, which has the same length than x. And I would like to extract from these arrays the subarrays, corresponding to the range x1 -- x2, going from x1, to x2. The output array should have its first element starting from the element in x closest to x1 (and in the range defined by x1, x2), and then going towards x2, for all cases (decreasing or increading order for x, and x1>=x2 or x1<=x2). So here is the output I would like to get in 3 different simple examples: ### Increasing order in x, and x1 <= x2 : x = arange(0.,1.,0.1) x1 = 0.1 x2 = 0.55 ### the output I would like is simply: array([ 0.1, 0.2, 0.3, 0.4, 0.5]) ### decreasing order in x, and x1 <= x2 : x = arange(0.,-1.,-0.1) x1 = -0.55 x2 = -0.1 ### I would like is then: array([ -0.5, -0.4, -0.3, -0.2, -0.1]) ### decreasing order in x, and x1 >= x2 : x = arange(0.,-1.,-0.1) x1 = -0.1 x2 = -0.55 ### I would like is then: array([ -0.1, -0.2, -0.3, -0.4, -0.5]) etc.... And it should work if both x1, and x2 are outside the given range provided in x (output should be an empty array) Note that I also need to extract the corresponding subarray from the data array (same indices as the one I extract from x). I hope this is clear. It is a very simple problem but I cannot see a simple solution without involving lots of stupid "if". It would be great if these "if" statements were hidden in some efficient numpy tricks/functions. thanks for any input. Eric

2 4

cpuinfo fails to recognize Core2 cpu on linux
by Nadav Horesh 28 Dec '06

28 Dec '06

System: gentoo linux (64 bits) on Core2 Duo. scipy fails to install since cpuinfo idendified the cpu as i686. Solution: I changed the linux_cpuinfo._is_Nocona() method to: def _is_Nocona(self): # return self.is_PentiumIV() and self.is_64bit() return re.match(r'Intel.*?Core.*\b', self.info[0]['model name']) is not None I am not sure that this solution is general enough. Here is my "cat /proc/cpuinfo" result: processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz stepping : 6 cpu MHz : 1596.000 cache size : 4096 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm bogomips : 4797.90 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual power management: processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 15 model name : Intel(R) Core(TM)2 CPU 6600 @ 2.40GHz stepping : 6 cpu MHz : 2394.000 cache size : 4096 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 fpu : yes fpu_exception : yes cpuid level : 10 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm syscall nx lm constant_tsc pni monitor ds_cpl vmx est tm2 cx16 xtpr lahf_lm bogomips : 4795.32 clflush size : 64 cache_alignment : 64 address sizes : 36 bits physical, 48 bits virtual Nadav.

1 0

Time series: lag function
by Vincent Nijs 27 Dec '06

27 Dec '06

I am tryin to convert some of my time-series code written in Ox to scipy/numpy (e.g., unit root tests, IRFs, cointegration, etc). Two key functions I need for this are 'lag' and 'diff'. 'diff' is available but 'lag' is apparently not. Below is my attempt at a lag function. I tried to be somewhat consistent with the diff function which is part of numpy (also listed for convenience). It seems to work fine for a 2-d array but not for a 1-d or 3-d array (see tests at bottom of email). I'd appreciate any suggestions you may have. Thanks, Vincent from numpy import * def diff(a, n=1, axis=-1): """Calculate the nth order discrete difference along given axis. """ if n == 0: return a if n < 0: raise ValueError, 'order must be non-negative but got ' + repr(n) a = asanyarray(a) nd = len(a.shape) slice1 = [slice(None)]*nd slice2 = [slice(None)]*nd slice1[axis] = slice(1, None) slice2[axis] = slice(None, -1) slice1 = tuple(slice1) slice2 = tuple(slice2) if n > 1: return diff(a[slice1]-a[slice2], n-1, axis=axis) else: return a[slice1]-a[slice2] def lag(a, n=1, lag_axis=0, concat_axis=1): """ Calculate the nth order discrete lag along given axis. Note: axis=-1 means 'last dimension'. This is the default for the diff function. However, the first dimension (0) may be preferred for time-series analysis. """ a = asanyarray(a) n = ravel(n) # convert input to an array nmax = n.max() # determines length of array to be returned nd = len(a.shape) # number of dimentions in array a s = [slice(None)]*nd # lag for 1st element in n s[lag_axis] = slice(nmax-n[0],-n[0]) ret_a = a[tuple(s)] # array to be returned # lags for other elements in n for i in n[1:]: s[lag_axis] = slice(nmax-i,-i) ret_a = concatenate((ret_a,a[tuple(s)]), concat_axis) return ret_a # testing lag function # test 1 data = arange(10) print "=" * 30 print "test 1 - data" print data print "\nlag 2" print lag(data,2) print "\nlag 1,2,3" print lag(data,range(1,4)) print "=" * 30 + "\n" # test 2 data = arange(10) data = vstack((data,data)).T print "=" * 30 print "test 2 - data" print data print "\nlag 2" print lag(data,2) print "\nlag 1,2,3" print lag(data,range(1,4)) print "=" * 30 + "\n" # test 3 data = arange(10) data = vstack((data,data)).T data = dstack((data,data)) print "=" * 30 print "test 3 - data" print data print "\nlag 2" print lag(data,2) print "\nlag 1,2,3," print lag(data,range(1,4)) print "=" * 30 + "\n"

2 2

which fft I should use
by Gennan Chen 27 Dec '06

27 Dec '06

Hi! all, There are so many fft routines in Scipy/Numpy. Does anyone know which one should be used officially? Gen

2 1

Small fix to Numexpr getType()
by Ivan Vilata i Balaguer 27 Dec '06

27 Dec '06

Hi all, According to Travis' advice in a prevoius thread (see http://www.mail-archive.com/numpy-discussion%40scipy.org/msg00442.html), I have modified the ``compiler.getType()`` function in Numexpr so that it uses the ``dtype.kind`` attribute instead of ``issubclass()``. The patch is attached. By the way, what is the proper casing of "Numexpr"? :) :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ Cárabos Coop. V. V V Enjoy Data ""

1 0

slow numpy.clip ?
by David Cournapeau 26 Dec '06

26 Dec '06

Hi, When trying to speed up some matplotlib routines with the matplotlib dev team, I noticed that numpy.clip is pretty slow: clip(data, m, M) is slower than a direct numpy implementation (that is data[data<m] = m; data[data>M] = M; return data.copy()). My understanding is that the code does the same thing, right ? Below, a small script which shows the difference (twice slower for a 8000x256 array on my workstation): import numpy as N #========================== # To benchmark imshow alone #========================== def generate_data_2d(fr, nwin, hop, len): nframes = 1.0 * fr / hop * len return N.random.randn(nframes, nwin) def bench_clip(): m = -1. M = 1. # 2 minutes (120 sec) of sounds @ 8 kHz with 256 samples with 50 % overlap data = generate_data_2d(8000, 256, 128, 120) def clip1_bench(data, niter): for i in range(niter): blop = N.clip(data, m, M) def clip2_bench(data, niter): for i in range(niter): data[data<m] = m data[data<M] = M blop = data.copy() clip1_bench(data, 10) clip2_bench(data, 10) if __name__ == '__main__': # test clip import hotshot, hotshot.stats profile_file = 'clip.prof' prof = hotshot.Profile(profile_file, lineevents=1) prof.runcall(bench_clip) p = hotshot.stats.load(profile_file) print p.sort_stats('cumulative').print_stats(20) prof.close() cheers, David

9 29

Different numpy.int64 in 64 bit platforms?
by Ivan Vilata i Balaguer 26 Dec '06

26 Dec '06

I have come across this strange behaviour of NumPy 1.0.1 under a 64 bit AMD Opteron: >>> import numpy >>> numpy.__version__ '1.0.1' >>> numpy.dtype(int).type <type 'numpy.int64'> >>> numpy.dtype(int).type is numpy.int64 True >>> numpy.dtype('int64').type <type 'numpy.int64'> >>> numpy.dtype('int64').type is numpy.int64 True >>> numpy.dtype(long).type <type 'numpy.int64'> >>> numpy.dtype(long).type is numpy.int64 # strange, but ok False >>> issubclass(numpy.dtype(long).type, numpy.int64) # what? False I.e. the NumPy type used for ``long`` is not the same than for ``int`` or explicit ``'int64'``, not even an instance. Is this a bug or is this kind of type comparisons discouraged? Thanks! :: Ivan Vilata i Balaguer >qo< http://www.carabos.com/ Cárabos Coop. V. V V Enjoy Data ""

2 1