[Numpy-discussion] Attempting to sub-class NumArray

Todd Miller jmiller at stsci.edu
Sun Sep 7 04:09:02 EDT 2003 

On Sat, 2003-09-06 at 11:25, Colin J. Williams wrote:
> Todd Miller wrote:
> 
> >I think the problem you are having is caused by the assumption that
> >instance initialization is equivalent to an unordered assignment of all
> >instance attributes.  That's not true because the private attributes are
> >interrelated;  assigning "shape" affects "_shape" and "_strides",
> >_itemsize must be known before _strides can be computed from shape, and
> >so on. 
> >
> It would be helpful if you could let us know what assignment ordering 
> would be appropriate, or, if there were information about the 
> interactions, perhaps we could work these things out for ourselves.

In truth,  I don't think the order is the issue, so let's drop that. 
All I can say is that somehow special.__init__() is creating
inconsistent _shape and _strides.  I also noticed it creating and
assigning bound methods, which you probably don't want to do.

> As we've discussed before, there is little information available on 
> private objects and their roles.  Even to have the source code of some 
> of the C code might be useful.  I wasn't able to spot the relevant C 
> code among the CVS directories.

The source code is all there.

Look in Include/numarray/numarray.h, Src/_ndarraymodule.c, and
Src/_numarraymodule.c, and maybe Src/libnumarraymodule.c.

> However, the assumption would appear to apply to the crude 
> initialization of 'special'.  The problem which is being reported seems 
> to be connected with the function setshape in generic.py.  This is far 
> from the unordered assignment, although I recognize that ther might be 
> some connection.

I'm not sure where and how, but I think the problem is in your dir()
loop.   So, instead, try this:

self.__setstate__(arr.__getstate__())

That ensures that you're only copying the data attributes and appears to
work.

Todd

> Colin W.
> 
> >
> >On Fri, 2003-09-05 at 20:32, Colin J. Williams wrote:
> >  
> >
> >>Todd,
> >>
> >>Thanks again for your prompt response.
> >>Todd Miller wrote:
> >>    
> >>
> >>>Hi Colin,
> >>>
> >>>I haven't looked at this in detail,  but special.__init__() is making me
> >>>queasy.  
> >>>      
> >>>
> >>We don't call __init__ directly, but only when we construct the
> >>instance, a,  of the class special.
> >>    
> >>
> >>>Why doesn't it call NumArray.__init__()?
> >>>      
> >>>
> >>Because we have data to install in the newly created a.  We go off to
> >>the factory function, numarray.array, to create the array, which we
> >>then specialize.
> >>
> >>I don't see how NumArray.__init__ would help.  This cumbersome
> >>procedure seems to work OK in most respects, but fails to display
> >>properly.
> >>
> >>I don't follow the code sufficiently well to be sure, but my hunch is
> >>that the problem lies with setShape.  It fails to update _strides.
> >>
> >>Colin W.
> >>    
> >>
> >>>Todd
> >>>
> >>>On Fri, 2003-09-05 at 15:03, Colin J. Williams wrote:
> >>>  
> >>>      
> >>>
> >>>>When ravel() is used on an instance of NumArray, the _strides attribute 
> >>>>is recalculated correctly.
> >>>>When ravel() is used on an instance of special, a sub-class of NumArray, 
> >>>>the _strides attribute keeps its old value.  This appears to cause the 
> >>>>display problem which is reported.
> >>>>
> >>>>The essentials of the code used are below.
> >>>>
> >>>>To investigate this problem, two print statements were added to the 
> >>>>method setshape in generic.py:
> >>>>
> >>>>            if newnelements == nelements:
> >>>>                self._shape = tuple(shape)
> >>>>                print 'In setShape self._shape:', self._shape
> >>>>                self._strides = self._stridesFromShape()
> >>>>                print 'In setShape self._strides:', self._strides
> >>>>            else:
> >>>>                raise ValueError("New shape is not consistent with the
> >>>>    old shape")
> >>>>
> >>>>It seems that only the first of these print statements is executed with 
> >>>>special, but both are executed with NumArray.  In the second case, the 
> >>>>output is not in the expected sequence.
> >>>>
> >>>>I would appreciate any suggestion as to a workaround.
> >>>>
> >>>>Colin W,
> >>>>
> >>>>Grief when attempting to sub-class NumArray
> >>>>
> >>>>Code used:
> >>>>
> >>>># Try to sub-class
> >>>>class special(N.NumArray):
> >>>>  def __init__(self, data= None, shape= None, eType= None):
> >>>>##                         eType= _nt.Any or AnyType not acceptable %%
> >>>>    arr= N.array(sequence= data, shape= shape, type= eType)
> >>>>    for attr in dir(arr):
> >>>>##      This is a longwinded way of setting up special
> >>>>##      There must be a better way                     %%
> >>>>##      Perhaps we should use generic._view
> >>>>##      It's not documented                 TRY IT LATER
> >>>>##      There is no need to transfer methods!
> >>>>      if attr[0] == '_' and attr[1] != '_':
> >>>>        print attr
> >>>>        exec 'self.' + attr + '= ' 'arr.' + attr
> >>>>        exec 'print self.' + attr
> >>>>    pass
> >>>>
> >>>>a= special(data= [1, 2, 3, 4], shape= (2, 2))
> >>>>a= N.array([1, 2, 3, 4], shape= (2, 2))
> >>>>print 'In ts a._strides:', a._strides
> >>>>b= _gen.ravel(a)
> >>>>print 'In ts b._strides:', b._strides       #  <<< Unchanged with 
> >>>>special, OK with NumArray
> >>>>print 'b:', b
> >>>>
> >>>>
> >>>>
> >>>>USING THE SUBCLASS special
> >>>> >C:\Progra~1\Python23\pythonw -u ts.py
> >>>>In ts a._strides: (8, 4)
> >>>>In setShape self._shape: (4,)
> >>>>In ts b._strides: (8, 4)
> >>>>b: In setShape self._shape: (4,)
> >>>>Traceback (most recent call last):
> >>>>  File "ts.py", line 37, in ?
> >>>>    print 'b:', b
> >>>>  File 
> >>>>"C:\PROGRA~1\Python23\lib\site-packages\numarray\numarraycore.py", line 
> >>>>618, in __str__
> >>>>    MAX_LINE_WIDTH, PRECISION, SUPPRESS_SMALL, ' ', "")
> >>>>  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
> >>>>line 176, in array2string
> >>>>    separator, prefix)
> >>>>  File "C:\PROGRA~1\Python23\lib\site-packages\numarray\arrayprint.py", 
> >>>>line 138, in _array2string
> >>>>    max_str_len = max(len(str(max_reduce(data))),
> >>>>libnumarray.error: maximum_Int32_reduce: access beyond buffer. offset=27 
> >>>>buffersize=16
> >>>> >Exit code: 1
> >>>>__________________________________________________________
> >>>>
> >>>># USING NumArray
> >>>>
> >>>>In setShape self._shape: (2, 2)
> >>>>In ts a._strides: (8, 4)
> >>>>In setShape self._shape: (4,)
> >>>>In ts b._strides: (4,)
> >>>>b: In setShape self._shape: (4,)
> >>>>[1 2 3 4]
> >>>> >Exit code: 0
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>-------------------------------------------------------
> >>>>This sf.net email is sponsored by:ThinkGeek
> >>>>Welcome to geek heaven.
> >>>>http://thinkgeek.com/sf
> >>>>_______________________________________________
> >>>>Numpy-discussion mailing list
> >>>>Numpy-discussion at lists.sourceforge.net
> >>>>https://lists.sourceforge.net/lists/listinfo/numpy-discussion
> >>>>    
> >>>>        
> >>>>
-- 
Todd Miller <jmiller at stsci.edu>

More information about the NumPy-Discussion mailing list