Mailman 3 May 2002 - NumPy-Discussion

Some problems with Numeric documenation
by Edward C. Jones 16 May '02

16 May '02

Here are some documentation problems with Numeric (20.2.0). I use the html form of the documentation. 1. descr->elsize is not documented. 2. The documentation for PyArray_FromDimsAndData says This function should only be used to access global data that will never be freed (like FORTRAN common blocks). The document writer copied this from the source code internal documentation. As far as I can tell, this is wrong. I suggest The object returned by this function contains a pointer to a pre-existing block of data. If you delete the data before the Python reference count has dropped to zero, you will have a dangling pointer which is usually a disaster. One safe way to do this is to Py_DECREF the object in the same C function where it was created. 3. By a semi-automated application of grep and Python, I have found the following functions which appear to be in the C API and are not documented. Note that PyArray_FromDimsAndData and PyArray_CopyArray can be combined to do the C level equivalent of arr[a:b,c:d] = something PyArray_ArgMax multiarraymodule.c:extern PyObject *PyArray_ArgMax(PyObject *op) { PyArray_ArgSort multiarraymodule.c:extern PyObject *PyArray_ArgSort(PyObject *op) { PyArray_BinarySearch multiarraymodule.c:extern PyObject *PyArray_BinarySearch(PyObject *op1, PyObject *op2) { PyArray_CONTIGUOUS arrayobject.c:#define PyArray_CONTIGUOUS(m) (ISCONTIGUOUS(m) ? Py_INCREF(m), m : \ PyArray_Choose multiarraymodule.c:extern PyObject *PyArray_Choose(PyObject *ip, PyObject *op) { PyArray_Concatenate multiarraymodule.c:extern PyObject *PyArray_Concatenate(PyObject *op) { PyArray_Converter arrayobject.c:extern int PyArray_Converter(PyObject *object, PyObject **address) { PyArray_CopyArray arrayobject.c:int PyArray_CopyArray(PyArrayObject *dest, PyArrayObject *src) { PyArray_CopyObject arrayobject.c:int PyArray_CopyObject(PyArrayObject *dest, PyObject *src_object) { PyArray_Correlate multiarraymodule.c:extern PyObject *PyArray_Correlate(PyObject *op1, PyObject *op2, int mode) { PyArray_FromDimsAndDataAndDescr arrayobject.c:PyObject *PyArray_FromDimsAndDataAndDescr(int nd, int *d, PyArray_FromScalar arrayobject.c:PyObject *PyArray_FromScalar(PyObject *op, int type) { PyArray_InnerProduct multiarraymodule.c:extern PyObject *PyArray_InnerProduct(PyObject *op1, PyObject *op2) { PyArray_Item arrayobject.c:extern PyObject * PyArray_Item(PyObject *op, int i) { PyArray_NBYTES arrayobject.h:#define PyArray_NBYTES(mp) ((mp)->descr->elsize * PyArray_SIZE(mp)) PyArray_Put arrayobject.c:extern PyObject *PyArray_Put(PyObject *self0, PyObject *indices0, PyArray_PutMask arrayobject.c:extern PyObject *PyArray_PutMask(PyObject *self0, PyObject *mask0, PyArray_Repeat multiarraymodule.c:extern PyObject *PyArray_Repeat(PyObject *aop, PyObject *op, int axis) { PyArray_Resize arrayobject.c:static PyObject * PyArray_Resize(PyArrayObject *self, PyObject *shape) { PyArray_Sort multiarraymodule.c:extern PyObject *PyArray_Sort(PyObject *op) { PyArray_TYPES arrayobject.h:enum PyArray_TYPES { PyArray_CHAR, PyArray_UBYTE, PyArray_SBYTE, PyArray_ToList arrayobject.c:static PyObject *PyArray_ToList(PyObject *self) { PyArray_Transpose multiarraymodule.c:extern PyObject *PyArray_Transpose(PyArrayObject *ap, PyObject *op) { PyArray_compare_lists arrayobject.c:extern int _PyArray_compare_lists(int *l1, int *l2, int n) { arrayobject.c:extern int _PyArray_compare_lists(int *l1, int *l2, int n) { multiarraymodule.c:extern PyObject *PyArray_Sort(PyObject *op) {

1 0

Pyfort version 8.0a1 in CVS
by Paul F Dubois 16 May '02

16 May '02

Pyfort version 8.0a1 is now in CVS for testing. A new user interface helps scientists create packages. Packages can be uninstalled. A GUI interface is provided for creating new packages and managing their compilation and installation. Documentation for the new features will be available shortly at pyfortran.sf.net.

1 0

Noise Simulation
by Hartley, Ed 14 May '02

14 May '02

Hi interesting discussion please could you quote the complete reference to this > I have based my routine on the document "Discrete simulation of power low > noise", IEEE International Frequency Control Symposium 27-29 > Regards Ed Hartley Research Fellow Computing Department Lancaster University Lancaster UK LA1 4YR Phone +44 (0) 1524 593675 Fax +44 (0) 1524 593608

3 2

real_fft
by Johan Fredrik Øhman 13 May '02

13 May '02

Hi there. I have a problem related to the fft routine in numpy. I'm trying to port a short C program to python, however it has turned out to be a little more complicated than initally thought. I'm not expecting anybody to look at the whole programs, so I have just cut out the important part (however, the complete source is included at the bottom of this mail. The program is creating colored noise) The problem is (most likely) that the C program uses a library called "Numerical Recipes". In this library there is a function called realft(). I don't know these FFT functions all that well, but there is a distinct difference from the NR (Numerical Recipies) realft() and the FFT.real_fft() function of Numerical Python. This is best illustrated by an example: Assume a list/array of 1024 integers. If you put this array through FFT.real_fft() you get a 513 long array as a result. The NR realft() gives a 2048 long array. Now, since C cannot deal with complex numbers it has to use two entries for each number. Still the array is much larger than the Numpy version. Anybody know why ? wfb and hfb are equal length. Is this a legal way to convolve using FFT.real_fft() ? wfb = FFT.real_fft(wfb) hfb = FFT.real_fft(hfb) for i in range(0, len(wfb)): wfb[i] = wfb[i] * hfb[i] wfb = FFT.inverse_real_fft(wfb) -- JFØ ==================== PYTHON ==================== import math, FFT, RNG, sys __gausian = RNG.NormalDistribution(0, 1) gausian = RNG.CreateGenerator(0, __gausian) def noiseGen(points, X, Qd, b): mhb = -b / 2.0 Qd = math.sqrt(Qd) # Deviation of the noise hfb = [0] * points wfb = [0] * points hfb[0] = 1.0 wfb[0] = Qd * gausian.ranf() for i in range(1, len(wfb)): # Generate hk coefficients hfb[i] = hfb[i-1]/float(i) * (i-1 + mhb) # Fille wk with white noise wfb[i] = Qd * gausian.ranf() wfb = FFT.real_fft(wfb) hfb = FFT.real_fft(hfb) print hfb # Multiply the complex vectors # Convolation for i in range(0, len(wfb)): wfb[i] = wfb[i] * hfb[i] wfb = FFT.inverse_real_fft(wfb) for i in range(0,len(wfb)): X[i] += wfb[i] if __name__ == '__main__': X = [0] * (2**10) noiseGen(2**10, X, 1, -2) for i in X: print i ========================== C Program ========================== #include <stdlib.h> #include <stdio.h> #include <string.h> #include <math.h> #include "c/numrec.h" #include "c/NRutil.h" void f_beta(int n_pts, float X[], float Q_d, float b, int *idum){ int i,nn; float *hfb,*wfb; float mhb,wr,wi; nn = n_pts * 2; mhb = -b / 2.0; Q_d = sqrt(Q_d); hfb = vector(1,nn); wfb = vector(1,nn); hfb[1] = 1.0; wfb[1] = Q_d * gasdev(idum); for(i=2; i<=n_pts; i++){ hfb[i] = hfb[i-1]*(mhb+(float)(i-2))/((float)(i-1)); wfb[i] = Q_d * gasdev(idum); } for(i=n_pts;i <= nn; i++){ hfb[i] = 0.0; wfb[i] = 0.0; } realft(hfb, n_pts,1); realft(wfb, n_pts,1); wfb[1]=wfb[1]*hfb[1]; wfb[2]=wfb[2]*hfb[2]; for(i=3; i<= nn; i += 2){ wr = wfb[i]; wi = wfb[i+1]; wfb[i] = wr*hfb[i]-wi*hfb[i+1]; wfb[i+1] = wr*hfb[i+1]+wi*hfb[i]; } realft(wfb, n_pts, -1); for(i=1; i <=n_pts;i++){ X[i] += wfb[i]/((float)n_pts); } free_vector(hfb,1,nn); free_vector(wfb,1,nn); } int main(){ int length22; int i; int idum = -210310212; float *X; X = vector(1,1024); f_beta(1024, X, 1, -2, &idum); for(i=1;i<=1024;i++){ printf("%f\n",X[i]); } return 0; } -- Johan Fredrik Øhman

2 2

tiny bug in MLab.py
by Charles G Waldman 10 May '02

10 May '02

>>> import Numeric >>> import MLab >>> >>> a=Numeric.ones(4) >>> b=Numeric.ones(4) >>> >>> print MLab.corrcoef(a,b) Traceback (most recent call last): File "<stdin>", line 1, in ? File "MLab.py", line 230, in corrcoef c = cov(x, y) File "MLab.py", line 237, in cov if y != None: m = concatenate((m,y)) TypeError: function not supported for these types, and can't coerce to supported types This is happening because of some comparisons of array objects to None. You can't say "a != None" for array objects. (Maybe you should be able to?) Anyhow, the following patch corrects the misbehavior: --- MLab.py.orig Fri May 10 16:12:52 2002 +++ MLab.py Fri May 10 16:36:42 2002 @@ -34,7 +34,7 @@ """eye(N, M=N, k=0, typecode=None) returns a N-by-M matrix where the k-th diagonal is all ones, and everything else is zeros. """ - if M == None: M = N + if M is None: M = N if type(M) == type('d'): typecode = M M = N @@ -46,7 +46,7 @@ """tri(N, M=N, k=0, typecode=None) returns a N-by-M matrix where all the diagonals starting from lower left corner up to the k-th are all ones. """ - if M == None: M = N + if M is None: M = N if type(M) == type('d'): typecode = M M = N @@ -197,7 +197,7 @@ """trapz(y,x=None,axis=-1) integrates y along the given dimension of the data array using the trapezoidal rule. """ - if x == None: + if x is None: d = 1.0 else: d = diff(x,axis=axis) @@ -234,7 +234,7 @@ def cov(m,y=None): m = asarray(m) mu = mean(m) - if y != None: m = concatenate((m,y)) + if y is not None: m = concatenate((m,y)) sum_cov = 0.0 for v in m: sum_cov = sum_cov+multiply.outer(v,v)

1 0

numarray on Cygwin
by Jochen Küpper 07 May '02

07 May '02

I am trying to run numarray on Cygwin. (Where Numeric seems to work fine.) In order to compile it I had to change the use of round to rint, as Cygwin's C library (newlib) doesn't have round. The patch is attached. When I use it I don't get far, though: ,---- | > python | Python 2.2 (#1, Dec 31 2001, 15:21:18) | [GCC 2.95.3-5 (cygwin special)] on cygwin | Type "help", "copyright", "credits" or "license" for more information. | >>> import numarray as na | >>> x = na.array([1.0, 2.0]) | >>> x | Traceback (most recent call last): | File "<stdin>", line 1, in ? | File "/usr/lib/python2.2/site-packages/numarray/numarray.py", line 591, in __repr__ | MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ', ', 1) | File "/usr/lib/python2.2/site-packages/numarray/arrayprint.py", line 86, in array2string | format, item_length = _floatFormat(data, precision, suppress_small) | File "/usr/lib/python2.2/site-packages/numarray/arrayprint.py", line 140, in _floatFormat | non_zero = numarray.abs(numarray.compress(numarray.not_equal(data, 0), data)) | File "/usr/lib/python2.2/site-packages/numarray/ufunc.py", line 474, in __call__ | cfunc, ufargs, scalar) | File "/usr/lib/python2.2/site-packages/numarray/ufunc.py", line 312, in _doit | apply(cfunc, ufargs) | TypeError: 'PyCObject' object is not callable | >>> `---- If I print out the type of cfunc down in _doit, I get (the expected) ,---- | <type 'PyCObject'> `---- Any ideas, probably some hints how to proceed? Greetings, Jochen -- University of North Carolina phone: +1-919-962-4403 Department of Chemistry phone: +1-919-962-1579 Venable Hall CB#3290 (Kenan C148) fax: +1-919-843-6041 Chapel Hill, NC 27599, USA GnuPG key: 44BCCD8E

2 3

import Numeric fails with user-supplied __import__?
by Mr. Bawkbugawk 06 May '02

06 May '02

Hey Folks, using the recently released python 2.1.3 with version 20.3 of the Numeric package, we are getting this behavior: (tommy@mace)/u0/tommy$ python2 Python 2.1.3 (#1, Apr 22 2002, 16:14:11) [C] on irix6 Type "copyright", "credits" or "license" for more information. >>> import ihooks >>> ihooks.install() >>> import Numeric Segmentation fault this is happening on both irix and linux. I checked the release notes for Numeric 20.1 but didn't see any mention of import bugs being fixed. a little bit of investigation shows that it isn't even ihooks that causes this, it is any user-supplied __import__: blah.py: import __builtin__ class blah: def __init__( self ): self.real_importer = __builtin__.__import__ __builtin__.__import__ = self.my_import def my_import( self, name, globs={}, locs={}, fromlist=[] ): return self.real_importer( name, globs, locs, fromlist ) _blah = blah() 859:tpelle@parro > python2 Python 2.1.3 (#1, Apr 19 2002, 21:22:42) [GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-81)] on linux2 Type "copyright", "credits" or "license" for more information. >>> import blah >>> import Numeric Segmentation fault (core dumped) anyone out there seen this before? any guess if its Numeric's problem or perhaps a bug deeper down in python's import architecture? any and all clues appreciated, thanks!

1 0

two tricky operations
by Pete Shinners 02 May '02

02 May '02

help, i need help formatting some data in Numeric arrays. i can't quite seem to make it happen. here they come... (beware) 1: i want to create a simple 3d array, so that accessing it with any two indices gives me the same result as those two indices.. myarray[1,2] == 1,2 myarray[4,4] == 4,4 i would assume this just involves mixing two arange()'s together, but i can't seem to get them meshed together correctly. actually, this is something i think i'd want quite a bit, it'd be real real nice if arange could take multiple axes. :] i thought this might be a good "fromfunction" situation, but i couldn't get that to go either. 2: i have a 3d array with image data (RGBs). i want to flatten only the first two dimensions so i have an array of RGB values. this one's got me completely stumped. i'm hoping there's just some function i'm not seeing in the docs, but i'd be just as happy with a bit of Numeric magic. thanks, please, and all that good stuff!

2 2