NumPy-Discussion June 2004

numpy-discussion@python.org

44 participants
64 discussions

printing character array object
by Faheem Mitha 23 Jun '04

23 Jun '04

Hi, it's me again. This time I am having problems with printing out a character array object (attached in cPickle format). I don't understand what the error message means. If you load the >>> import cPickle >>> cPickle.dump(foo,open('foo.save','w')) >>> foo [tons of error messages terminating in] libnumarray.error: copy1bytes: access beyond buffer. offset=6 buffersize=6 Looks like the problem is with numarray. The character array is 100x20 and I can't see any intrinsic problem with printing it out. >>> foo.getshape() Out[73]: (100, 20) Thanks for any clarification. Faheem.

1 0

Java version for numarray?
by Francesc Alted 22 Jun '04

22 Jun '04

Hi, I'm using numarray objects basically like data containers for fast I/O purposes in the context of pytables. I'm thinking now to port part of my code to Java, and I'd like to use Jython to easy the transition. I've seen that a there is a port of Numeric to Jython (http://jnumerical.sourceforge.net/), and I thought that I could use these Numeric objects as containers. Unfortunately, there are a couple of objects that pytables relies on, namely RecArray and CharArray, that are not supported by Numeric. My questions are: - I think that both RecArray and CharArray modules are written in pure python, so the porting to Jython should be relatively easy, provided I'm successful making them to use Numeric objects instead of NumArray objects. What do you think? Would that be feasible? - Is there any effort going on in order to have an implementation of numarray ready for use in Jython? Thanks, -- Francesc Alted

2 1

convolve2d
by Michel Sanner 22 Jun '04

22 Jun '04

Hello, I started using numarray for doing 2D convolutions on images. I noticed that import numarray.examples.convolve.high_level as convolve convolve.Convolve2d(kernel, in, out) only works on square images. For images that are not square I get lots of noise in the background. Also I was wondering is using the high_level API is most efficient? Currently my image is a Numeric array (grabbed from the OpenGL frame buffer) which I convert to a numarray to do the convolution and back to a Numeric array. In the future I hope to completely replace Numeric by numarray. Thanks for any input -Michel -- ----------------------------------------------------------------------- o / Michel F. Sanner Ph.D. The Scripps Research Institute o Associate Professor Department of Molecular Biology \ 10550 North Torrey Pines Road o Tel. (858) 784-2341 La Jolla, CA 92037 / Fax. (858) 784-2860 o sanner(a)scripps.edu http://www.scripps.edu/~sanner -----------------------------------------------------------------------

2 2

median filtering question -- masked arrays
by Russell E Owen 22 Jun '04

22 Jun '04

Does anyone have a hint for applying a median filter to a masked 2D array in numarray? I've been using the kludge of extracting a normal array with the masked data replaced by the overall median of the array and median filtering that (with the appropriate nd_image method). But this is a crude approximation and I'd like to do better -- preferably without coding my own median filter in C. -- Russell

1 0

numarray.concatenate for character arrays
by Faheem Mitha 22 Jun '04

22 Jun '04

Dear People, Is the function numarray.concatenate supposed to work for character arrays? It doesn't for me. Do I need to write my own? Thanks in advance. Please cc me, I'm not subscribed. Faheem. In [17]: foo Out[17]: CharArray([['T', 'T'], ['C', 'A']]) In [18]: bar Out[18]: CharArray([['G', 'G'], ['G', 'G']]) In [23]: numarray.concatenate((foo,bar)) --------------------------------------------------------------------------- error Traceback (most recent call last) /home/faheem/wc/corrmodel/trunk/<console> /usr/lib/python2.3/site-packages/numarray/generic.py in concatenate(arrs, axis) 1018 arrs = map(_nc.asarray, arrs) 1019 if axis == 0: -> 1020 return _concat(arrs) 1021 else: 1022 return swapaxes(_concat([swapaxes(m,axis,0) for m in arrs]), axis, 0) /usr/lib/python2.3/site-packages/numarray/generic.py in _concat(arrs) 1000 convType = ufunc._maxPopType(arrs) 1001 except TypeError: -> 1002 dest = arrs[0]._clone(shape=destShape) 1003 else: 1004 dest = arrs[0].__class__(shape=destShape, type=convType) /usr/lib/python2.3/site-packages/numarray/generic.py in _clone(self, shape) 783 def _clone(self, shape): 784 c = self.copy() --> 785 c.resize(shape) 786 return c 787 /usr/lib/python2.3/site-packages/numarray/generic.py in resize(self, shape, *args) 854 self[offset:offset+olen] = self[0:olen] 855 offset += olen --> 856 self[offset:nlen] = self[0:nlen-offset] 857 else: # zero fill resized zero-length numarray 858 self[:] = 0 /usr/lib/python2.3/site-packages/numarray/strings.py in _copyFrom(self, arr) 217 me = self._byteView() 218 if self._itemsize <= arr._itemsize: --> 219 me[:] = it[..., :self._itemsize] 220 else: 221 me[...,:it._shape[-1]] = it error: copy1bytes: access beyond buffer. offset=8 buffersize=8

2 1

RE: [Numpy-discussion] lock in LinearAlgebra when embedding
by Sebastien.deMentendeHorne＠electrabel.com 22 Jun '04

22 Jun '04

It looks like it is a similar problem. However, it is not yet resolved as the patch is from may/2003 and I have this problem in the latest Numeric version. Moreover, my problem lies in the embedded/not embedded difference when running the same script. In fact, I notice it when using Python embedded in Excel via VBA. Testing the same script from the command-line is OK while from Excel it hangs. And the python/numeric I use is the "native windows" version (i.e. compiled with VC++ and not cygwin/mingw). So, Numeric embedded works if lapack_lite is compile with mingw32 but NOT with VC++. Looks like this lapack interface is delicate... Seb -----Original Message----- From: Michiel Jan Laurens de Hoon [mailto:mdehoon@ims.u-tokyo.ac.jp] Sent: mardi 22 juin 2004 14:21 To: Sebastien.deMentendeHorne(a)electrabel.com Cc: numpy-discussion(a)lists.sourceforge.net Subject: Re: [Numpy-discussion] lock in LinearAlgebra when embedding This may be related to patch 732520. The problem is that lapack_lite is not compiled correctly by setup.py; one part of it should be compiled without optimization, but is compiled with. On many platforms, this doesn't cause any problems, but on Cygwin (and therefore probably also on MinGW) the eigenvalues function hangs as a result. You can get more information by looking up the patch on sourceforge. This issue has come up a couple of times. Maybe we should fix the Numeric/numarray source code with this patch or something similar? --Michiel, U Tokyo. Sebastien.deMentendeHorne(a)electrabel.com wrote: > Hi, > > There is a nasty behaviour in the LinearAlgebra package under windows NT. > > When I run the following script from the command-line (python2.3 on windows > Enthought edition but I had the same problem with the original python2.3 + > Numeric 23.3), it returns without problem. > ##################### > from LinearAlgebra import eigenvalues > from Numeric import array > a = array([[3,4],[1,6]]) > print eigenvalues(a) > ##################### > Output > [ 2. 7.] > > However, when I evaluate the same script embedded in C, it hangs while > consuming 100% of the CPU: > /* C Code */ > #include <Python.h> > > int > main(int argc, char *argv[]) > { > Py_Initialize(); > PyRun_SimpleString("from LinearAlgebra import eigenvalues\nfrom Numeric > import array\na = array([[3,4],[1,6]])\nprint eigenvalues(a)\n"); > > Py_Finalize(); > return 0; > } > /* end of C code */ > > I compile the code with Mingw > gcc embed.c -Ic:\python23\include -Lc:\python23 -lpython23 > and gcc -v gives > > Reading specs from > C:/Python23/Enthought/MingW/bin/../lib/gcc-lib/mingw32/3.2.3/specs > Configured with: ../gcc/configure --with-gcc --with-gnu-ld --with-gnu-as > --host=mingw32 --target=mingw32 --prefix=/mingw > --enable-threads --disable-nls --enable-languages=c++,f77,objc > --disable-win32-registry --disable-shared --enable-sjlj- > exceptions > Thread model: win32 > gcc version 3.2.3 (mingw special 20030504-1) > > I must recompile lapack_lite from source with Mingw in order to get the > correct behaviour. > > Any hint on a cleaner solution (not recompile lapack_lite whenever I install > a new version of Numeric). > > Seb > > > ------------------------------------------------------- > This SF.Net email sponsored by Black Hat Briefings & Training. > Attend Black Hat Briefings & Training, Las Vegas July 24-29 - > digital self defense, top technical experts, no vendor pitches, > unmatched networking opportunities. Visit www.blackhat.com > _______________________________________________ > Numpy-discussion mailing list > Numpy-discussion(a)lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/numpy-discussion > > -- Michiel de Hoon, Assistant Professor University of Tokyo, Institute of Medical Science Human Genome Center 4-6-1 Shirokane-dai, Minato-ku Tokyo 108-8639 Japan http://bonsai.ims.u-tokyo.ac.jp/~mdehoon

1 0

lock in LinearAlgebra when embedding
by Sebastien.deMentendeHorne＠electrabel.com 22 Jun '04

22 Jun '04

Hi, There is a nasty behaviour in the LinearAlgebra package under windows NT. When I run the following script from the command-line (python2.3 on windows Enthought edition but I had the same problem with the original python2.3 + Numeric 23.3), it returns without problem. ##################### from LinearAlgebra import eigenvalues from Numeric import array a = array([[3,4],[1,6]]) print eigenvalues(a) ##################### Output [ 2. 7.] However, when I evaluate the same script embedded in C, it hangs while consuming 100% of the CPU: /* C Code */ #include <Python.h> int main(int argc, char *argv[]) { Py_Initialize(); PyRun_SimpleString("from LinearAlgebra import eigenvalues\nfrom Numeric import array\na = array([[3,4],[1,6]])\nprint eigenvalues(a)\n"); Py_Finalize(); return 0; } /* end of C code */ I compile the code with Mingw gcc embed.c -Ic:\python23\include -Lc:\python23 -lpython23 and gcc -v gives Reading specs from C:/Python23/Enthought/MingW/bin/../lib/gcc-lib/mingw32/3.2.3/specs Configured with: ../gcc/configure --with-gcc --with-gnu-ld --with-gnu-as --host=mingw32 --target=mingw32 --prefix=/mingw --enable-threads --disable-nls --enable-languages=c++,f77,objc --disable-win32-registry --disable-shared --enable-sjlj- exceptions Thread model: win32 gcc version 3.2.3 (mingw special 20030504-1) I must recompile lapack_lite from source with Mingw in order to get the correct behaviour. Any hint on a cleaner solution (not recompile lapack_lite whenever I install a new version of Numeric). Seb

2 1

Unpickled numarray types unsable?
by Francesc Alted 21 Jun '04

21 Jun '04

Hi, I'm experiencing problems when using a numarray datatype after pickle/unpickle it: >>> import numarray >>> import pickle >>> c=pickle.dumps(numarray.Float64) >>> t=pickle.loads(c) >>> t Float64 >>> na=numarray.array(type=t, shape=(2,)) Traceback (most recent call last): File "<stdin>", line 1, in ? File "/usr/local/lib/python2.3/site-packages/numarray/numarraycore.py", line 321, in array return NumArray(buffer=sequence, shape=shape, type=type) RuntimeError: _numarray_init: can't get typeno for type Perhaps this is a bug and the typeno attribute is missed when reconstructing the type? Cheers, -- Francesc Alted

2 1

Subclassing NumArray: _PROTOTYPE flag issue
by Nadav Horesh 21 Jun '04

21 Jun '04

For a simulation project I am working on I've subclasses ArrayType. I was able to do much of my intentions until in one place when I tried to make an array from a list of arrays I got an error message: . . . File "/usr/local/lib/python2.3/site-packages/numarray/numarraycore.py", line 325, in array return fromlist(sequence, type, shape) File "/usr/local/lib/python2.3/site-packages/numarray/numarraycore.py", line 212, in fromlist a = a.astype(type) File "/usr/local/lib/python2.3/site-packages/numarray/numarraycore.py", line 630, in astype retarr = self.__class__(buffer=None, shape=self._shape, type=type) TypeError: __init__() got an unexpected keyword argument 'buffer' The analysis of the code showed that: 1. The NumArray class method definitions depends on the _PROTOTYPE flag 2. The post-mortem debugging showed that when the error flagged, the value of the variable _PROTOTYPE was 0 In a stand alone script there was no problem to do the list-> array conversion: >>> import numarray as N >>> import NumImage as NI # My module with the derived class >>> a = N.arange(4) >>> ia = NI.Cimage(N.arange(4)) # CImage is a derivative of NumImage >>> a array([0, 1, 2, 3]) >>> ia Cimage([0, 1, 2, 3]) >>> N.array([a+i for i in range(3)]) array([[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5]]) >>> N.array([ia+i for i in range(3)]) # OK here, but failed as a part of a complex script Cimage([[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5]]) My questions are: 1. Is this flag is in use? If I set it to 0 will I be able to derive a class from the "C code"? 2. Any intelligent solution? Nadav.

3 2

arrayfns.histogram does it exist in numarray?
by Jean-Bernard Addor 21 Jun '04

21 Jun '04

Hello ! I am analysing data in hdf files with Numeric. For different reasons, including better support of hdf files (huge datasets), I try to switch to numarray. I uses a lot the arrayfns.histogram function of Numeric, does something similar exist in nummarray? See you, Jean-Bernard

3 2

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

NumPy-Discussion June 2004