python user defined type

Hi all, I'm rather new to the list so maybe the question is well known but from my researches on the web and list archive, I did not find a clear explanation. I would like to create numpy array with my own (python) datatype, so I tried the obvious solution: from numpy import * class Unit(object): def __init__(self,value=0.0): self.value = value def __float__(self): return self.value def __repr__(self): return str(self.value) a = array([[Unit(1), Unit(2)], [Unit(3), Unit(4)]]) print a print a.dtype.name print a[0,0].value #a[0,0] = 0 #print a[0,0].value a = array (array([[1,2],[3,4]]), dtype=Unit) print a print a.dtype print a[0,0].value but the last print make python to raise an AttributeError stating that int object do not have a 'value' attribute. I guess I missed something concerning numpy object data type. Do I need to implement some special methods in Unit to make numpy happy ? Also, the commented line (a[0,0] = 0) makes the item to become an int while I need to set the value of the item instead, is that possible ? Nicolas

Nicolas Rougier wrote:
I would like to create numpy array with my own (python) datatype, so I tried the obvious solution:
from numpy import * class Unit(object): def __init__(self,value=0.0): self.value = value def __float__(self): return self.value def __repr__(self): return str(self.value)
a = array (array([[1,2],[3,4]]), dtype=Unit)
the dtype argument is designed to take a numpy type object, not an arbitrary class -- what you want in this case is dtype=numpy.object, which is what you did before. I'm a surprised this didn't raise an error, but it looks like you got an object array, but you objects you gave it are python integers. All python classes are "objects" as far an numpy is concerned. The numpy dtypes are a description of a given number of bytes in memory -- python classes are stored as a pointer to the python object. (and you really don't want to use "import *")
Also, the commented line (a[0,0] = 0) makes the item to become an int while I need to set the value of the item instead, is that possible ?
a[0,0] = Unit(0) You're setting the [0,0]th element to an object, you need to give it the object you want, the literal "0" is a python integer with the value zero. numpy arrays of python objects act a whole lot like other python containers. What would you expect from : a = [1,2,3,4] a list of integers, no? or a = [Unit(1), Unit(2)] # a list of Units.. then # a[0] = 3 now a list with the integer3 in the zeroth position, and a Unit in the 1st. You did it right the first time: a = array([[Unit(1), Unit(2)], [Unit(3), Unit(4)]]) though you need to be careful building object arrays of nested lists -- numpy won't unnecessarily figure out how do d-nest it. You might want to do:
import numpy as np a = np.empty((2,2), np.object) a array([[None, None], [None, None]], dtype=object) a[:,:] = [[Unit(1), Unit(2)], [Unit(3), Unit(4)]] a array([[1, 2], [3, 4]], dtype=object)
One more note: class Unit(object): def __init__(self,value=0.0): self.value = value def __float__(self): return self.value def __repr__(self): return str(self.value) __repr__ really should be something like: def __repr__(self): return "Unit(%g)%self.value" eval(repr(Object)) == Object, ideally, plus it'll be easier to debug if you can see what it is. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

Thanks for the precise explanation that make things clearer. In fact, the example I gave was mainly to illustrate my question in the quickest way. Concerning the dtype argument during array creation, I thought it was here for somehow controlling the type of array elements. For example, if I use a "regular" numpy array (let's say a float array), I cannot set an item to a string value (it raises a ValueError: setting an array element with a sequence). So what would be the best way to use numpy arrays with "foreign" types (or is it possible at all) ? I've designed the "real" Unit in C++ and exported it to python (via boost and shared pointers) and I would like to create array of such Units (in fact, I also created an array-like class but I would prefer to use directly the real array interface to benefit from the great work of numpy instead of re-inventing the wheel). Ideally, I would like to be able to write z = N.array (a, dtype=Unit) and would expect numpy to make a copy of the array by calling my type with each element of a. Then, if my type accepts the argument during creation, everything's fine, else it raises an error. Nicolas On Thu, 2008-07-10 at 13:43 -0700, Christopher Barker wrote:
Nicolas Rougier wrote:
I would like to create numpy array with my own (python) datatype, so I tried the obvious solution:
from numpy import * class Unit(object): def __init__(self,value=0.0): self.value = value def __float__(self): return self.value def __repr__(self): return str(self.value)
a = array (array([[1,2],[3,4]]), dtype=Unit)
the dtype argument is designed to take a numpy type object, not an arbitrary class -- what you want in this case is dtype=numpy.object, which is what you did before. I'm a surprised this didn't raise an error, but it looks like you got an object array, but you objects you gave it are python integers. All python classes are "objects" as far an numpy is concerned. The numpy dtypes are a description of a given number of bytes in memory -- python classes are stored as a pointer to the python object.
(and you really don't want to use "import *")
Also, the commented line (a[0,0] = 0) makes the item to become an int while I need to set the value of the item instead, is that possible ?
a[0,0] = Unit(0)
You're setting the [0,0]th element to an object, you need to give it the object you want, the literal "0" is a python integer with the value zero.
numpy arrays of python objects act a whole lot like other python containers. What would you expect from :
a = [1,2,3,4]
a list of integers, no?
or
a = [Unit(1), Unit(2)]
# a list of Units..
then
# a[0] = 3
now a list with the integer3 in the zeroth position, and a Unit in the 1st.
You did it right the first time:
a = array([[Unit(1), Unit(2)], [Unit(3), Unit(4)]])
though you need to be careful building object arrays of nested lists -- numpy won't unnecessarily figure out how do d-nest it. You might want to do:
import numpy as np a = np.empty((2,2), np.object) a array([[None, None], [None, None]], dtype=object) a[:,:] = [[Unit(1), Unit(2)], [Unit(3), Unit(4)]] a array([[1, 2], [3, 4]], dtype=object)
One more note:
class Unit(object): def __init__(self,value=0.0): self.value = value def __float__(self): return self.value def __repr__(self): return str(self.value)
__repr__ really should be something like: def __repr__(self): return "Unit(%g)%self.value"
eval(repr(Object)) == Object, ideally, plus it'll be easier to debug if you can see what it is.
-Chris

Nicolas Rougier wrote:
Concerning the dtype argument during array creation, I thought it was here for somehow controlling the type of array elements. For example, if I use a "regular" numpy array (let's say a float array), I cannot set an item to a string value (it raises a ValueError: setting an array element with a sequence).
Yes, but numpy is designed primarily for numeric types: ints, floats, etc. It can also be used for custom types that are essentially like C structs (see recarray). The key is that a dtype desribes a data type in terms of bytes and that they represent -- It can not be a python type. The only way to use arbitrary python types is a object array, which you've discovered, but then numpy doesn't know any thing about the objects, other than that they are python objects.
So what would be the best way to use numpy arrays with "foreign" types (or is it possible at all) ? I've designed the "real" Unit in C++ and exported it to python (via boost and shared pointers) and I would like to create array of such Units
If your type is a C++ class, then it may be possible, with some C hackary to get numpy to understand it, but you're getting beyong my depth here -- also I doubt that you'd get the full features like array math and all anyway -- that's all set up for basic numeric types. Maybe others will have some idea, but I think you're pushing what numpy is capable of.
(in fact, I also created an array-like class but I would prefer to use directly the real array interface to benefit from the great work of numpy instead of re-inventing the wheel).
What operations do you expect to perform with these arrays of Units? -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

My Unit class is supposed to represent a neuron that can be linked to any other unit. The neuron itself is merely a (float) potential that can vary along time under the influence of other units and learning. I gather these units into groups which are in fact 2D matrix of units. Currently, I implemented the Unit and Group and I "talk" with numpy through an attribute of groups which represent all available potentials. Finally my group is like a simple 2d matrix of float but I need to have an underlying object to perform computation on each Unit at each time step. Currently I'm able to write something like:
group = Unit()*[2,2] group.potentials = numpy.zeros([2,2]) print group.potentials [[ 0. 0.] [ 0. 0.]] group[0,0].potential = 1 [[ 1. 0.] [ 0. 0.]]
Nicolas On Thu, 2008-07-10 at 16:30 -0700, Christopher Barker wrote:
Nicolas Rougier wrote:
Concerning the dtype argument during array creation, I thought it was here for somehow controlling the type of array elements. For example, if I use a "regular" numpy array (let's say a float array), I cannot set an item to a string value (it raises a ValueError: setting an array element with a sequence).
Yes, but numpy is designed primarily for numeric types: ints, floats, etc. It can also be used for custom types that are essentially like C structs (see recarray). The key is that a dtype desribes a data type in terms of bytes and that they represent -- It can not be a python type.
The only way to use arbitrary python types is a object array, which you've discovered, but then numpy doesn't know any thing about the objects, other than that they are python objects.
So what would be the best way to use numpy arrays with "foreign" types (or is it possible at all) ? I've designed the "real" Unit in C++ and exported it to python (via boost and shared pointers) and I would like to create array of such Units
If your type is a C++ class, then it may be possible, with some C hackary to get numpy to understand it, but you're getting beyong my depth here -- also I doubt that you'd get the full features like array math and all anyway -- that's all set up for basic numeric types.
Maybe others will have some idea, but I think you're pushing what numpy is capable of.
(in fact, I also created an array-like class but I would prefer to use directly the real array interface to benefit from the great work of numpy instead of re-inventing the wheel).
What operations do you expect to perform with these arrays of Units?
-Chris

Nicolas Rougier wrote:
My Unit class is supposed to represent a neuron that can be linked to any other unit. The neuron itself is merely a (float) potential that can vary along time under the influence of other units and learning.
gather these units into groups which are in fact 2D matrix of units.
If you don't want to get into numpy internals, or write your own array classes, my thought is to re-think the structure, and rather than have an array of Unit objects, have a subclass of numpy array that represents the group, that is an array floats, plus whatever meta information you need. -Chris -- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Thu, Jul 10, 2008 at 16:45, Nicolas Rougier <Nicolas.Rougier@loria.fr> wrote:
Ideally, I would like to be able to write
z = N.array (a, dtype=Unit)
and would expect numpy to make a copy of the array by calling my type with each element of a. Then, if my type accepts the argument during creation, everything's fine, else it raises an error.
Just use dtype=object and make any copies you need yourself. Encapsulate this logic in a function if you need it frequently. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco
participants (3)
-
Christopher Barker
-
Nicolas Rougier
-
Robert Kern