Mailman 3 September 2009 - NumPy-Discussion

subclassing
by Keith Goodman 29 Sep '09

29 Sep '09

I ran across a problem while using numpy. But the problem is more of python problem. I hope I am not too far off topic. I have a class and a subclass: class myclass: def __init__(self, x): self.x = x def __getitem__(self, index): if type(index) is slice: x = self.x[index] return myclass(x) else: raise IndexError, 'Only slicing is allowed in this example.' def calc(self): print 'myclass' def __repr__(self): return self.x.__repr__() class mysubclass(myclass): def calc(self): print 'mySUBclass' I can make an instance of mysubclass: >> sub = mysubclass(np.array([1,2,3])) >> sub array([1, 2, 3]) >> sub.calc() mySUBclass The problem occurs when I slice: >> sub_slice = sub[1:] >> sub_slice.calc() myclass # <--- Oops. I'd like this to be mySUBclass Slicing causes sub to change from mysubclass to myclass. It does so because __getitem__ returns myclass(x). I could make a __setitem__ method in mysubclass that returns mysubclass(x), but I have many such methods with the same problem (not shown in the code above). Is there another solution?

2 2

fixed_pt some progress and a question
by Neal Becker 29 Sep '09

29 Sep '09

I'm starting with a pure python implementation and have some progress. AFAICT, the only approach is to subclass ndarray and add the properties and behaviors I need. I ran into one issue though. In my function 'as_double', I need to get to the underlying 'int' array to pass to ldexp. I tried using view, but it silently fails: In [40]: obj Out[40]: fixed_pt_array([ 0, 32, 64, 96, 128]) In [41]: obj.view (int) Out[41]: fixed_pt_array([ 0, 32, 64, 96, 128]) How can I get at the underlying int array to pass to ldexp? Here is the prototype code (far from complete!) import numpy as np def rnd (x, frac_bits, _max): x1 = x >> (frac_bits-1) if (x1 == _max): return x1 >> 1 else: return (x1+1) >> 1 def shift_left (x, bits): return x << bits def shift_right (x, bits): return x >> bits def shiftup_or_rnddn (x, bits, _max, rnd_policy): if (bits > 0): return shift_left (x, bits) elif (bits < 0): return rnd_policy (x, -bits, _max) else: return x def clip (x, _min, _max): if x > _max: return _max elif x < _min: return _min else: return x class fixed_pt (object): def get_max(self): if self.is_signed: return (~(self.base_type(-1) << (self.total_bits-1))) else: return (~(self.base_type (-1) << self.total_bits)) def get_min(self): if self.is_signed: return ((self.base_type(-1) << (self.total_bits-1))) else: return 0 def __init__ (self, int_bits, frac_bits, val, scale=True, base_type=int, rnd_policy=rnd, overflow_policy=clip, is_signed=True): self.is_signed = is_signed self.int_bits = int_bits self.frac_bits = frac_bits self.base_type = base_type self.total_bits = int_bits + frac_bits self.rnd_policy = rnd_policy self._max = self.get_max () self._min = self.get_min () self.overflow_policy = overflow_policy if scale: self.val = self.overflow_policy (self.base_type (shiftup_or_rnddn (val, frac_bits, self._max, self.rnd_policy)), self._min, self._max) def as_double (self): return np.ldexp (self.val, -self.frac_bits) def as_base (self): return shiftup_or_rnddn (self.val, -self.frac_bits, self._max, self.rnd_policy) def __repr__(self): return "[%s <%s,%s>]" % (self.val, self.int_bits, self.frac_bits) def get_max(is_signed, base_type, total_bits): if is_signed: return (~(base_type(-1) << (total_bits-1))) else: return (~(base_type (-1) << total_bits)) def get_min(is_signed, base_type, total_bits): if is_signed: return ((base_type(-1) << (total_bits-1))) else: return 0 class fixed_pt_array(np.ndarray): def __new__(cls, input_array, int_bits, frac_bits, scale=True, base_type=int, rnd_policy=rnd, overflow_policy=clip, is_signed=True): # Input array is an already formed ndarray instance # We first cast to be our class type obj = np.asarray(input_array, dtype=base_type).view(cls) # add the new attribute to the created instance obj.int_bits = int_bits obj.frac_bits = frac_bits obj.rnd_policy = rnd_policy obj.overflow_policy = overflow_policy obj.is_signed = is_signed obj.scale = scale obj.base_type = base_type obj.total_bits = int_bits + frac_bits obj._max = get_max(is_signed, base_type, obj.total_bits) obj._min = get_min(is_signed, base_type, obj.total_bits) if scale: def _scale (val): return overflow_policy (base_type (shiftup_or_rnddn (val, frac_bits, obj._max, rnd_policy)), obj._min, obj._max) vecfunc = np.vectorize (_scale) obj = vecfunc (obj) # Finally, we must return the newly created object: return obj def __array_finalize__(self,obj): # reset the attribute from passed original object if hasattr (obj, 'int_bits'): self.int_bits = obj.int_bits self.frac_bits = obj.frac_bits self.is_signed = obj.is_signed self.base_type = obj.base_type self.total_bits = obj.total_bits self._max = obj._max self._min = obj._min ## self._max = get_max(self.is_signed, self.base_type, self.total_bits) ## self._min = get_min(self.is_signed, self.base_type, self.total_bits) # We do not need to return anything ## def __getitem__ (self, index): ## return fp.fixed_pt_int64_clip (self.int_bits, self.frac_bits, int(np.ndarray.__getitem__(self, index))) def as_double (self): def _as_double (x): print type(x) return np.ldexp (x, -self.frac_bits) return _as_double (self.view (self.base_type)) fp = fixed_pt (5, 5, 1) arr = np.arange(5,dtype=int) obj = fixed_pt_array(arr, int_bits=5, frac_bits=5) print type(obj)

3 4

The problem with zero dimesnsional array
by yogesh karpate 29 Sep '09

29 Sep '09

Dear All, I'm facing a bog problem in following . the code snippet is as follows #################### % Compute the area indicator################################### for kT in range(leftbound,rightbound): # Here the left bound and rightbound both are indexing array is cutlevel = sum(s[(kT-ptwin):(kT+ptwin)],0)/(ptwin*2+1) corsig = s[(kT-swin+1):kT]-cutlevel areavalue1 =sum((corsig),0) #print areavalue.size print leftbound, rightbound Tval=areavalue1[leftbound:rightbound] Everything works fine till areavalue1, then whenever I try to access the Tval=areavalue1[leftbound:rightbound] it says IndexError: invalid index to scalar variable.. When i try to access areavalue1[0] it gives me entire array but for areavalue1[2:8]..it gives the same error . Thanx in advance.. Regards ymk

2 1

MFDatasets and NetCDF4
by George Nurser 29 Sep '09

29 Sep '09

Hi, I hope this is the right place to ask this. I've found the MFDataset works well in reading NetCDF3 files, but it appears that it doesn't work at present for NetCDF4 files. Is this an inherent problem with the NetCDF4 file structure, or would it be possible to implement the MFDataset for NetCDF4 files sometime? It would be very useful. --George Nurser.

5 8

setup.py does not use the correct library
by Peng Yu 29 Sep '09

29 Sep '09

On Mon, Sep 28, 2009 at 4:44 PM, Robert Kern <robert.kern(a)gmail.com> wrote: > On Mon, Sep 28, 2009 at 16:40, Peng Yu <pengyu.ut(a)gmail.com> wrote: >> I attached the script that I run for build and the build output. I >> think that setup.py doesn't use the correct python library. But I'm >> not sure why. Would you please help me figure out what the problem is? > > Setting $LDFLAGS to be empty is also incorrect. Simply do not set > $LDFLAGS or $CPPFLAGS at all. > > [And please do not Cc: me. I read the list.] Even if I don't set $LDFLAGS in my build script, I still have LDFLAGS set in my ~/.bash_profile. Here is the my build script and the build output. It still gives me errors. What is wrong with it? $ cat ~/notes/install/numpy/build #!/usr/bin/env bash #export LD_LBRARY_PATH= #export CPPFLAGS="-I$HOME/utility/linux/opt/Python-2.6.2/include/python2.6" #export LDFLAGS="-L$HOME/utility/linux/opt/Python-2.6.2/lib -lpython2.6" #export CPPFLAGS= #export LDFLAGS= export PATH=$HOME/.bin/misc:$HOME/utility/linux/opt/Python-2.6.2/bin:$HOME/utility/linux/opt/gcc-4.3.4/bin:/usr/local/bin:/usr/bin:/bin #echo $PATH gcc --version python --version python setup.py build --fcompiler=gnu95 | tee build_output ; sendmail.sh build_output $ tail -20 build_output /home/pengy/download/linux/python/numpy-1.3.0rc2/build/src.linux-x86_64-2.6/numpy/core/include/numpy/__multiarray_api.h:1007: undefined reference to `PyExc_RuntimeError' /home/pengy/download/linux/python/numpy-1.3.0rc2/build/src.linux-x86_64-2.6/numpy/core/include/numpy/__multiarray_api.h:1007: undefined reference to `PyErr_Format' build/temp.linux-x86_64-2.6/numpy/linalg/lapack_litemodule.o: In function `initlapack_lite': /home/pengy/download/linux/python/numpy-1.3.0rc2/numpy/linalg/lapack_litemodule.c:830: undefined reference to `PyErr_Print' /home/pengy/download/linux/python/numpy-1.3.0rc2/numpy/linalg/lapack_litemodule.c:830: undefined reference to `PyExc_ImportError' build/temp.linux-x86_64-2.6/numpy/linalg/lapack_litemodule.o: In function `_import_array': /home/pengy/download/linux/python/numpy-1.3.0rc2/build/src.linux-x86_64-2.6/numpy/core/include/numpy/__multiarray_api.h:977: undefined reference to `PyCObject_AsVoidPtr' /home/pengy/download/linux/python/numpy-1.3.0rc2/build/src.linux-x86_64-2.6/numpy/core/include/numpy/__multiarray_api.h:984: undefined reference to `PyExc_RuntimeError' /home/pengy/download/linux/python/numpy-1.3.0rc2/build/src.linux-x86_64-2.6/numpy/core/include/numpy/__multiarray_api.h:984: undefined reference to `PyErr_Format' /home/pengy/download/linux/python/numpy-1.3.0rc2/build/src.linux-x86_64-2.6/numpy/core/include/numpy/__multiarray_api.h:996: undefined reference to `PyExc_RuntimeError' /home/pengy/download/linux/python/numpy-1.3.0rc2/build/src.linux-x86_64-2.6/numpy/core/include/numpy/__multiarray_api.h:996: undefined reference to `PyErr_Format' build/temp.linux-x86_64-2.6/numpy/linalg/lapack_litemodule.o: In function `initlapack_lite': /home/pengy/download/linux/python/numpy-1.3.0rc2/numpy/linalg/lapack_litemodule.c:833: undefined reference to `PyDict_SetItemString' /home/pengy/download/linux/python/numpy-1.3.0rc2/numpy/linalg/lapack_litemodule.c:830: undefined reference to `PyErr_SetString' build/temp.linux-x86_64-2.6/numpy/linalg/python_xerbla.o: In function `xerbla_': /home/pengy/download/linux/python/numpy-1.3.0rc2/numpy/linalg/python_xerbla.c:35: undefined reference to `PyExc_ValueError' /home/pengy/download/linux/python/numpy-1.3.0rc2/numpy/linalg/python_xerbla.c:35: undefined reference to `PyErr_SetString' /home/pengy/utility/linux/opt/gcc-4.3.4/lib/gcc/x86_64-unknown-linux-gnu/4.3.4/libgfortranbegin.a(fmain.o): In function `main': /home/pengy/download/linux/gcc-4.3.4-build-20090905-220719/x86_64-unknown-linux-gnu/libgfortran/../../../gcc-4.3.4/libgfortran/fmain.c:21: undefined reference to `MAIN__' collect2: ld returned 1 exit status Regards, Peng

2 1

Re: [Numpy-discussion] setup.py does not use the correct library
by Robert Kern 28 Sep '09

28 Sep '09

On Mon, Sep 28, 2009 at 16:40, Peng Yu <pengyu.ut(a)gmail.com> wrote: > I attached the script that I run for build and the build output. I > think that setup.py doesn't use the correct python library. But I'm > not sure why. Would you please help me figure out what the problem is? Setting $LDFLAGS to be empty is also incorrect. Simply do not set $LDFLAGS or $CPPFLAGS at all. [And please do not Cc: me. I read the list.] -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco

1 0

setup.py does not use the correct library
by Peng Yu 28 Sep '09

28 Sep '09

I use the following command to build numpy-1.3.0rc2. But it seems not able to find the appropriate library files. Can somebody let me know how to make it use the correct ones? #####command export LD_LBRARY_PATH= export CPPFLAGS="-I$HOME/utility/linux/opt/Python-2.6.2/include/python2.6" export LDFLAGS="-L$HOME/utility/linux/opt/Python-2.6.2/lib -lpython2.6" export PATH=$HOME/utility/linux/opt/Python-2.6.2/bin:/usr/local/bin:/usr/bin:/bin python setup.py build --fcompiler=gnu95 #######error /home/pengy/download/linux/python/numpy-1.3.0rc2/build/src.linux-x86_64-2.6/numpy/core/include/numpy/__multiarray_api.h:996: undefined reference to `PyErr_Format' build/temp.linux-x86_64-2.6/numpy/linalg/lapack_litemodule.o: In function `initlapack_lite': /home/pengy/download/linux/python/numpy-1.3.0rc2/numpy/linalg/lapack_litemodule.c:833: undefined reference to `PyDict_SetItemString' /home/pengy/download/linux/python/numpy-1.3.0rc2/numpy/linalg/lapack_litemodule.c:830: undefined reference to `PyErr_SetString' build/temp.linux-x86_64-2.6/numpy/linalg/python_xerbla.o: In function `xerbla_': /home/pengy/download/linux/python/numpy-1.3.0rc2/numpy/linalg/python_xerbla.c:35: undefined reference to `PyExc_ValueError' /home/pengy/download/linux/python/numpy-1.3.0rc2/numpy/linalg/python_xerbla.c:35: undefined reference to `PyErr_SetString' /usr/lib/gcc/x86_64-redhat-linux/4.1.2/libgfortranbegin.a(fmain.o): In function `main': (.text+0xa): undefined reference to `MAIN__' collect2: ld returned 1 exit status

2 3

Deprecate poly1d and replace with Poly1d ?
by Charles R Harris 28 Sep '09

28 Sep '09

Because poly1d exports the __array__ interface, a design error IMHO that makes it play badly with the prospective chebyshev module. For example the following should convert from a Chebyshev series to a power series chebval([1,0,0], poly1d(1,0)) and it does if I make sure to pass the poly1d as object in an array, but to do that I have to check that it is an instance of poly1d and take special measures. I shouldn't have to do that, it violates duck typing. The more basic problem here is making poly1d look like an array, which it isn't. The array bit is an implementation detail and would be private in C++. with an as_array method to retrieve the details if wanted. Chuck

2 2

numpy.reshape() bug?
by Chris 28 Sep '09

28 Sep '09

I am trying to "collapse" two dimensions of a 3-D array, using reshape: (Pdb) dims = np.shape(trace) (Pdb) dims Out[2]: (1000, 4, 3) (Pdb) newdims = (dims[0], sum(dims[1:])) (Pdb) newdims Out[2]: (1000, 7) However, reshape seems to think I am missing something: (Pdb) np.reshape(trace, newdims) *** ValueError: total size of new array must be unchanged Clearly the total size of the new array *is* unchanged.

2 1

new array type in cython?
by Neal Becker 28 Sep '09

28 Sep '09

Has anyone attempted a new array type in cython? Any hints?

3 6