Mailman 3 - NumPy-Discussion

memory not being freed
by Mathew Yeates 26 Feb '02

26 Feb '02

Hi I'm having problems with garbage collection I wrote an extension which creates an array and returns it foo() { arr = (PyArrayObject *) PyArray_FromDims( ..... ret = Py_BuildValue("O", arr); return ret; } but now if I do while 1: a=foo() memory is never free'd. I've even tried explicitly calling gc.collect and adding del(a) after a=foo. Is the problem that Py_BuildValue increases the reference count? Mathew

4 4

Hilbert transform
by Nils Wagner 19 Feb '02

19 Feb '02

Hi, I am looking for a Numpy function that performs the Hilbert transform of a scalar time series. Thanks in advance. Nils Wagner

1 0

Re: [Numpy-discussion] memory-mapped Numeric arrays: arrayfrombuffer version 2
by kragen＠pobox.com 18 Feb '02

18 Feb '02

(I thought I had sent this mail on January 30, but I guess I was mistaken.) Eric Nodwell writes: > Since I have a 2.4GB data file handy, I thought I'd try this > package with it. (Normally I process this data file by reading > it in a chunk at a time, which is perfectly adequate.) Not > surprisingly, it chokes: Yep, that's pretty much what I expected. I think that adding code to support mapping some arbitrary part of a file should be fairly straightforward --- do you want to run the tests if I write the code? > File "/home/eric/lib/python2.2/site-packages/maparray.py", line 15, > in maparray > m = mmap.mmap(fn, os.fstat(fn)[stat.ST_SIZE]) > OverflowError: memory mapped size is too large (limited by C int) This error message's wording led me to something that was *not* what I expected. That's a sort of alarming message --- it suggests that it won't work on >2G files even on LP64 systems, where longs and pointers are 64 bits but ints are 32 bits. The comments in the mmap module say: The map size is restricted to [0, INT_MAX] because this is the current Python limitation on object sizes. Although the mmap object *could* handle a larger map size, there is no point because all the useful operations (len(), slicing(), sequence indexing) are limited by a C int. Horrifyingly, this is true. Even the buffer interface function arrayfrombuffer uses to get the size of the buffer return int sizes, not size_t sizes. This is a serious bug in the buffer interface, IMO, and I doubt it will be fixed --- the buffer interface is apparently due for a revamp soon at any rate, so little changes won't be welcomed, especially if they break binary backwards compatibility, as this one would on LP64 platforms. Fixing this, so that LP64 Pythons can mmap >2G files (their birthright!), is a bit of work --- probably a matter of writing a modified mmap() module that supports a saner version of the buffer interface (with named methods instead of a type object slot), and can't be close()d, to boot. Until then, this module only lets you memory-map files up to two gigs. > (details: Python 2.2, numpy 20.3, Pentium III, Debian Woody, Linux > kernel 2.4.13, gcc 2.95.4) My kernel is 2.4.13 too, but I don't have any large files, and I don't know whether any of my kernel, my libc, or my Python even support them. > I'm not a big C programmer, but I wonder if there is some way for > this package to overcome the 2GB limit on 32-bit systems. That > could be useful in some situations. I don't know, but I think it would probably require extensive code changes throughout Numpy. -- <kragen(a)pobox.com> Kragen Sitaker <http://www.pobox.com/~kragen/> The sages do not believe that making no mistakes is a blessing. They believe, rather, that the great virtue of man lies in his ability to correct his mistakes and continually make a new man of himself. -- Wang Yang-Ming

2 1

I solved my fortran equivalence to Python conversion
by Rob 16 Feb '02

16 Feb '02

Here is the code: ARL1=zeros((441),Complex) ARL2=zeros((341),Complex) ARL3=zeros((289),Complex) #DIMENSION ARL1(1), ARL2(1), ARL3(1) #EQUIVALENCE (ARL1,AR1), (ARL2,AR2), (ARL3,AR3), (XS2,XSA(2)), (YS3,YSA(3)) AR1=swapaxes(AR1,0,2) AR2=swapaxes(AR2,0,2) AR3=swapaxes(AR3,0,2) ARL1[1:]=ravel(AR1[1:,1:,1:]) ARL2[1:]=ravel(AR2[1:,1:,1:]) ARL3[1:]=ravel(AR3[1:,1:,1:]) -- The Numeric Python EM Project www.pythonemproject.com

1 0

Converting from FORTRAN Equivalence to a Numpy alternative
by Rob 16 Feb '02

16 Feb '02

I hoped there might be a combo FORTRAN/Numpy hacker that might help me with this. I have the following statements in a FORTRAN routine that I am converting to Numpy. I am wondering how to handle the EQUIVALENCE statement. As I understand it, ARL1[1] should be equal to the first item in the memory block for AR1. But not knowing how FORTRAN allocates arrays, its hard to tell what to do in Numpy. I tried ARL1=ravel(AR1) but that didn't work. Similarly ARL=AR[:,1,1} didn't work. I'm lost. Thanks in advance for any help. Rob. #COMMON /GGRID/ AR1(11,10,4),AR2(17,5,4),AR3(9,8,4),EPSCF,DXA(3),& # &DYA(3),XSA(3),YSA(3),NXA(3),NYA(3 #DIMENSION ARL1(1), ARL2(1), ARL3(1) #EQUIVALENCE (ARL1,AR1), (ARL2,AR2), (ARL3,AR3), (XS2,XSA(2)), (YS3,YSA(3)) -- The Numeric Python EM Project www.pythonemproject.com

1 0

Re: [Numpy-discussion] subsampling
by mathew＠fugue.jpl.nasa.gov 15 Feb '02

15 Feb '02

John J. Lee writes: > I think -- correct me if I'm wrong -- he was asking about interpolation. I think that only Mr Yeates knows for sure! I read the query a few times and I think he's looking for straight-ahead subsampling without interpolation. Yes, I was just asking about subsampling although I did start to think about doing bilinear interpolation. But I was hoping to subsample using built in Numpy functions and .... your solution fits the bill!! Thank you! Mathew

1 0

subsampling
by Mathew Yeates 15 Feb '02

15 Feb '02

Hi- Maybe this is a dumb question .... but.... how can I take a matrix and subsample it down to an arbitrary size? It seems that using slices is limited to constant integer step sizes while, if I'm converting between arbitrary sizes, I would like to subsample at a nonuniform rate. Am I missing something? Mathew

3 3

Proposal, recoded average.
by Paul F. Dubois 14 Feb '02

14 Feb '02

I recoded the new average function in both Numeric and MA. This version ensures that in the returned=1 case the weights returned will be of the same shape as the average. The new version is in CVS and labeled as 21.0b2. The average now allows weights that, rather than having to be the same size as the input, can be the same size as the input's axis'th dimension. This relieves the user of the rather tricky code needed to make an array of a given shape broadcasting a vector along a certain axis. This is a relatively frequent need and so I will write a separate routine for it soon. My proposed calling sequence is: def broadcast(v, axis=0, s) """Returns an array of the type of v and the shape s, such that setting the axis'th subscript to j results in an array whose values are all v[j]. """ Comments about this idea or the way I explain it would be welcome.

1 0

Simulating white noise with Numpy
by Nils Wagner 07 Feb '02

07 Feb '02

Hi, How can I simulate a stationary, white noise, Gaussian random process (zero mean and unit variance) with Numpy ? Nils

2 1

numpy installation on HP-UX
by SungPil Yoon 31 Jan '02

31 Jan '02

Hi, all I have been trying to install numpy. Python is installed in my home directory ~/Python-2.2 and when I do % python setup.py install in the directory ~/Numeric-20.3, I get the following error message distutils.errors.DistutilsPlatformError: invalid Python installation: unable to open /usr/local/lib/python2.2/config/Makefile (No such file or directory) Can anybody figure out what I am doing wrong? sungpil

1 0