Hi, it's me again. This time I am having problems with printing out a
character array object (attached in cPickle format). I don't understand
what the error message means.
If you load the
>>> import cPickle
[tons of error messages terminating in]
libnumarray.error: copy1bytes: access beyond buffer. offset=6 buffersize=6
Looks like the problem is with numarray. The character array is 100x20 and
I can't see any intrinsic problem with printing it out.
Out: (100, 20)
Thanks for any clarification.
I'm using numarray objects basically like data containers for fast I/O
purposes in the context of pytables. I'm thinking now to port part of my
code to Java, and I'd like to use Jython to easy the transition. I've seen
that a there is a port of Numeric to Jython
(http://jnumerical.sourceforge.net/), and I thought that I could use these
Numeric objects as containers. Unfortunately, there are a couple of objects
that pytables relies on, namely RecArray and CharArray, that are not
supported by Numeric.
My questions are:
- I think that both RecArray and CharArray modules are written in pure
python, so the porting to Jython should be relatively easy, provided I'm
successful making them to use Numeric objects instead of NumArray objects.
What do you think? Would that be feasible?
- Is there any effort going on in order to have an implementation of
numarray ready for use in Jython?
I started using numarray for doing 2D convolutions on images. I noticed
import numarray.examples.convolve.high_level as convolve
convolve.Convolve2d(kernel, in, out)
only works on square images. For images that are not square I get lots
of noise in the background.
Also I was wondering is using the high_level API is most efficient?
Currently my image is a Numeric
array (grabbed from the OpenGL frame buffer) which I convert to a
numarray to do the convolution
and back to a Numeric array. In the future I hope to completely replace
Numeric by numarray.
Thanks for any input
/ Michel F. Sanner Ph.D. The Scripps Research Institute
o Associate Professor Department of Molecular Biology
\ 10550 North Torrey Pines Road
o Tel. (858) 784-2341 La Jolla, CA 92037
/ Fax. (858) 784-2860
o sanner(a)scripps.edu http://www.scripps.edu/~sanner
Does anyone have a hint for applying a median filter to a masked 2D
array in numarray?
I've been using the kludge of extracting a normal array with the
masked data replaced by the overall median of the array and median
filtering that (with the appropriate nd_image method). But this is a
crude approximation and I'd like to do better -- preferably without
coding my own median filter in C.
Is the function numarray.concatenate supposed to work for character
arrays? It doesn't for me. Do I need to write my own? Thanks in
advance. Please cc me, I'm not subscribed.
In : foo
In : bar
In : numarray.concatenate((foo,bar))
error Traceback (most recent call
1018 arrs = map(_nc.asarray, arrs)
1019 if axis == 0:
-> 1020 return _concat(arrs)
1022 return swapaxes(_concat([swapaxes(m,axis,0) for m in
arrs]), axis, 0)
/usr/lib/python2.3/site-packages/numarray/generic.py in _concat(arrs)
1000 convType = ufunc._maxPopType(arrs)
1001 except TypeError:
-> 1002 dest = arrs._clone(shape=destShape)
1004 dest = arrs.__class__(shape=destShape,
/usr/lib/python2.3/site-packages/numarray/generic.py in _clone(self,
783 def _clone(self, shape):
784 c = self.copy()
--> 785 c.resize(shape)
786 return c
/usr/lib/python2.3/site-packages/numarray/generic.py in resize(self,
854 self[offset:offset+olen] = self[0:olen]
855 offset += olen
--> 856 self[offset:nlen] = self[0:nlen-offset]
857 else: # zero fill resized zero-length numarray
858 self[:] = 0
217 me = self._byteView()
218 if self._itemsize <= arr._itemsize:
--> 219 me[:] = it[..., :self._itemsize]
221 me[...,:it._shape[-1]] = it
error: copy1bytes: access beyond buffer. offset=8 buffersize=8
It looks like it is a similar problem. However, it is not yet resolved as
the patch is from may/2003 and I have this problem in the latest Numeric
Moreover, my problem lies in the embedded/not embedded difference when
running the same script.
In fact, I notice it when using Python embedded in Excel via VBA. Testing
the same script from the command-line is OK while from Excel it hangs.
And the python/numeric I use is the "native windows" version (i.e. compiled
with VC++ and not cygwin/mingw). So, Numeric embedded works if lapack_lite
is compile with mingw32 but NOT with VC++.
Looks like this lapack interface is delicate...
From: Michiel Jan Laurens de Hoon [mailto:email@example.com]
Sent: mardi 22 juin 2004 14:21
Subject: Re: [Numpy-discussion] lock in LinearAlgebra when embedding
This may be related to patch 732520. The problem is that lapack_lite is not
compiled correctly by setup.py; one part of it should be compiled without
optimization, but is compiled with. On many platforms, this doesn't cause
problems, but on Cygwin (and therefore probably also on MinGW) the
function hangs as a result. You can get more information by looking up the
This issue has come up a couple of times. Maybe we should fix the
Numeric/numarray source code with this patch or something similar?
--Michiel, U Tokyo.
> There is a nasty behaviour in the LinearAlgebra package under windows NT.
> When I run the following script from the command-line (python2.3 on
> Enthought edition but I had the same problem with the original python2.3 +
> Numeric 23.3), it returns without problem.
> from LinearAlgebra import eigenvalues
> from Numeric import array
> a = array([[3,4],[1,6]])
> print eigenvalues(a)
> [ 2. 7.]
> However, when I evaluate the same script embedded in C, it hangs while
> consuming 100% of the CPU:
> /* C Code */
> #include <Python.h>
> main(int argc, char *argv)
> PyRun_SimpleString("from LinearAlgebra import eigenvalues\nfrom Numeric
> import array\na = array([[3,4],[1,6]])\nprint eigenvalues(a)\n");
> return 0;
> /* end of C code */
> I compile the code with Mingw
> gcc embed.c -Ic:\python23\include -Lc:\python23 -lpython23
> and gcc -v gives
> Reading specs from
> Configured with: ../gcc/configure --with-gcc --with-gnu-ld --with-gnu-as
> --host=mingw32 --target=mingw32 --prefix=/mingw
> --enable-threads --disable-nls --enable-languages=c++,f77,objc
> --disable-win32-registry --disable-shared --enable-sjlj-
> Thread model: win32
> gcc version 3.2.3 (mingw special 20030504-1)
> I must recompile lapack_lite from source with Mingw in order to get the
> correct behaviour.
> Any hint on a cleaner solution (not recompile lapack_lite whenever I
> a new version of Numeric).
> This SF.Net email sponsored by Black Hat Briefings & Training.
> Attend Black Hat Briefings & Training, Las Vegas July 24-29 -
> digital self defense, top technical experts, no vendor pitches,
> unmatched networking opportunities. Visit www.blackhat.com
> Numpy-discussion mailing list
Michiel de Hoon, Assistant Professor
University of Tokyo, Institute of Medical Science
Human Genome Center
4-6-1 Shirokane-dai, Minato-ku
There is a nasty behaviour in the LinearAlgebra package under windows NT.
When I run the following script from the command-line (python2.3 on windows
Enthought edition but I had the same problem with the original python2.3 +
Numeric 23.3), it returns without problem.
from LinearAlgebra import eigenvalues
from Numeric import array
a = array([[3,4],[1,6]])
[ 2. 7.]
However, when I evaluate the same script embedded in C, it hangs while
consuming 100% of the CPU:
/* C Code */
main(int argc, char *argv)
PyRun_SimpleString("from LinearAlgebra import eigenvalues\nfrom Numeric
import array\na = array([[3,4],[1,6]])\nprint eigenvalues(a)\n");
/* end of C code */
I compile the code with Mingw
gcc embed.c -Ic:\python23\include -Lc:\python23 -lpython23
and gcc -v gives
Reading specs from
Configured with: ../gcc/configure --with-gcc --with-gnu-ld --with-gnu-as
--host=mingw32 --target=mingw32 --prefix=/mingw
--enable-threads --disable-nls --enable-languages=c++,f77,objc
--disable-win32-registry --disable-shared --enable-sjlj-
Thread model: win32
gcc version 3.2.3 (mingw special 20030504-1)
I must recompile lapack_lite from source with Mingw in order to get the
Any hint on a cleaner solution (not recompile lapack_lite whenever I install
a new version of Numeric).
I'm experiencing problems when using a numarray datatype after
>>> import numarray
>>> import pickle
>>> na=numarray.array(type=t, shape=(2,))
Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/local/lib/python2.3/site-packages/numarray/numarraycore.py", line 321, in array
return NumArray(buffer=sequence, shape=shape, type=type)
RuntimeError: _numarray_init: can't get typeno for type
Perhaps this is a bug and the typeno attribute is missed when reconstructing
For a simulation project I am working on I've subclasses ArrayType. I
was able to do much of my intentions until in one place when I tried to
make an array from a list of arrays I got an error message:
325, in array
return fromlist(sequence, type, shape)
212, in fromlist
a = a.astype(type)
630, in astype
retarr = self.__class__(buffer=None, shape=self._shape, type=type)
TypeError: __init__() got an unexpected keyword argument 'buffer'
The analysis of the code showed that:
1. The NumArray class method definitions depends on the _PROTOTYPE flag
2. The post-mortem debugging showed that when the error flagged, the
value of the variable _PROTOTYPE was 0
In a stand alone script there was no problem to do the list-> array
>>> import numarray as N
>>> import NumImage as NI # My module with the derived class
>>> a = N.arange(4)
>>> ia = NI.Cimage(N.arange(4)) # CImage is a derivative of NumImage
array([0, 1, 2, 3])
Cimage([0, 1, 2, 3])
>>> N.array([a+i for i in range(3)])
array([[0, 1, 2, 3],
[1, 2, 3, 4],
[2, 3, 4, 5]])
>>> N.array([ia+i for i in range(3)]) # OK here, but failed as a part
of a complex script
Cimage([[0, 1, 2, 3],
[1, 2, 3, 4],
[2, 3, 4, 5]])
My questions are:
1. Is this flag is in use? If I set it to 0 will I be able to derive
a class from the "C code"?
2. Any intelligent solution?
I am analysing data in hdf files with Numeric. For different reasons,
including better support of hdf files (huge datasets), I try to switch to
numarray. I uses a lot the arrayfns.histogram function of Numeric, does
something similar exist in nummarray?