Is the following the desired behaviour for setting recarray attributes? This seems to clash with the semantics for arrays. >>> from numpy import * >>> a = array([1,2,3]) >>> b = a.view([('x',int),('y',int),('z',int)]) >>> r = b.view(recarray) >>> b['x'] = 0 >>> r.y = 0 >>> a array([0, 2, 3]) >>> r.x array([0]) >>> r.y 0 Setting r.y creates a local variable in r.__dict__ which is then accessed rather than the array element referred to by fielddict. I am not sure how to fix this: trying to set an attribute that does not exist in an ndarray raises an error, so I figured that changing the second line of recarray.__setattr__ in numpy/core/records.py to def __setattr__(self, attr, val): try: return sb.ndarray.__setattr__(self, attr, val) would work, but this sets the variable in r.__dict__ (and I am not exactly sure where this is defined, so I can't comment further). Is there a simple solution like this or does one need to explicitly check '__dict__'. One could reverse the order and call 'setfield' if the value is in the 'fielddict' first and then revert to the base setter, but this would potentially hide other members like 'shape'. (Making this change does not break any tests however.) A test case for this would be: def check_recarray_setattr1(self): """Make sure __setattr__ sets array fields.""" a = array([1,2]) r = rec.fromarrays(a,names=['x','y']) b = a.view(r.dtype).view(recarray) b.x = 0 assert 0 == a[0] def check_recarray_setattr2(self): """Make sure shape attribute is not hidden.""" a = array([[1,2],[3,4]]) r = rec.fromarrays(a,names=['x','shape']) b = a.view(r.dtype).view(recarray) assert 3 == b[1,0][0] b.shape = (1,2) assert 3 == b[0,1][0] Michael. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
On Friday 27 October 2006 22:18, Michael McNeil Forbes wrote:
Is the following the desired behaviour for setting recarray attributes?
I ran into the same problem recently... What about modifying __setattr__ to the following ? def __setattr__(self, attr, val): fielddict = sb.ndarray.__getattribute__(self,'dtype').fields if attr in fielddict.keys(): return sb.ndarray.__setitem__(self, attr, val) try: return object.__setattr__(self, attr, val) except AttributeError: # Must be a fieldname raise AttributeError, "record array has no attribute %s" % attr The order is changed, as in Michael's suggestion, but that doesn't set any variable in the __dict__, nor does it mask anything. Instead of modifying the an attribute, we modify the field directly... ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642 _______________________________________________ Numpy-discussion mailing list Numpy-discussion@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/numpy-discussion
Michael McNeil Forbes wrote:
Is the following the desired behaviour for setting recarray attributes? This seems to clash with the semantics for arrays.
from numpy import * a = array([1,2,3]) b = a.view([('x',int),('y',int),('z',int)]) r = b.view(recarray) b['x'] = 0 r.y = 0 a array([0, 2, 3]) r.x array([0]) r.y 0
Setting r.y creates a local variable in r.__dict__ which is then accessed rather than the array element referred to by fielddict.
Hmm.... I know that the code was changed at some point a few months ago specifically to this behavior because of some concerns Perry, Chris (people at STScI) had. Originally, field names came first, but we changed it so they could set known attributes of a record array even if they were also field names. This may be an unintentional side-effect. So, let's not just change things again and create problems for them. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
On Saturday 28 October 2006 14:28, Travis Oliphant wrote:
Hmm.... I know that the code was changed at some point a few months ago specifically to this behavior because of some concerns Perry, Chris (people at STScI) had. Originally, field names came first, but we changed it so they could set known attributes of a record array even if they were also field names.
So that I could have a field named 'shape', and modifying r.shape would change the shape of the array, not the content of the field ? That makes sense, but isn't it a bit dangerous ? Shouldn't we have a list of reserved keywords (dtype, shape...) that would raise an exception if used as field names ? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
In article <200610281530.22929.pgmdevlist@gmail.com>,
Pierre GM
So that I could have a field named 'shape', and modifying r.shape would change the shape of the array, not the content of the field ?
That makes sense, but isn't it a bit dangerous ? Shouldn't we have a list of reserved keywords (dtype, shape...) that would raise an exception if used as field names ?
Since the field names can be used with regular arrays without any consequences, I think it would be bad to raise an exception with recarrays where there was no problem with regular arrays, and regular arrays should allow field names such as 'shape'. A note should be added to the recarray documentation, however, pointing out this pitfall. Perhaps a warning could be raised when a recarray is constructed with poor fieldname choices, but I would not raise an exception. In any case, I think is is dangerous to allow the fieldnames to hide standard array members. This would break the array interface for such recarrays causing many subtle problems for any algorithm that assumed the recarray to behave as a standard array: This code could be deeply buried. Instead, the user's code would behave 'strangely', but it is also the user's code that assigned bad fieldnames, and a warning would alert of this. Another concern that I found with the recarray interface is that the current implementation exhibits quite poor performance, but I am not sure how to fix this yet. Is there any documentation about the general design decisions behind recarrays? I think they could be very useful for simplifying code, but could use several improvements. Michael. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
On Saturday 28 October 2006 22:45, Michael McNeil Forbes wrote:
Since the field names can be used with regular arrays without any consequences, I think it would be bad to raise an exception with recarrays where there was no problem with regular arrays, and regular arrays should allow field names such as 'shape'.
I agree to a certain extent: as switching from regular arrays to recarrays is straightforward, any field can potentially can be accessed as an attribute and this can brings some serious surprises. If an exception is too restrictive, a warning should at least be raised (just as the new warnings about dividing by zeros) when an array is created with field names in a exclude list. As a starter, the exclude list could be most (all ?) of the basic attributes/methods of a regular ndarray, along with 'fields', 'mask'...
Another concern that I found with the recarray interface is that the current implementation exhibits quite poor performance, but I am not sure how to fix this yet.
Well, there's a lot of __getattribute__ access: that tends to slow things down. But wouldn't checking whether the fields are in the 'exclude' list degrade the performance even more ? ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
In article <4543A153.4030505@ieee.org>,
Travis Oliphant
Hmm.... I know that the code was changed at some point a few months ago specifically to this behavior because of some concerns Perry, Chris (people at STScI) had. Originally, field names came first, but we changed it so they could set known attributes of a record array even if they were also field names.
This may be an unintentional side-effect. So, let's not just change things again and create problems for them.
Were any test cases generated for this? Changing the code did not break anything: if this is important, a test should probably be added. Michael. ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
Michael McNeil Forbes wrote:
In article <4543A153.4030505@ieee.org>, Travis Oliphant
wrote: Hmm.... I know that the code was changed at some point a few months ago specifically to this behavior because of some concerns Perry, Chris (people at STScI) had. Originally, field names came first, but we changed it so they could set known attributes of a record array even if they were also field names.
This may be an unintentional side-effect. So, let's not just change things again and create problems for them.
Were any test cases generated for this? Changing the code did not break anything: if this is important, a test should probably be added.
You are right. In fact there are lots of tests that need to be added. Stefan as been doing a great job of regression testing. I've committed some fixes to this particular issue that allows setattr to be used to set field names. Now 1) The standard setattr is tried first 2) If it fails, then the field-dictionary is checked to see if there is a field with that name. a) If not, then the error created by 1 is raised b) if the name is a field, then an attempt is made to set the field with that name 3) If the standard setattr succeeds then either i) a built-in attribute was correctly set or ii) a new attribute was created and attached to the dictionary. a) If the attribute is not in the fields, then return b) Otherwise, see if this name was in the .__dict__ before the call (if it's not then we just added it so delete it). i) If the deletion succeeds then set set the field ii If the deletion fails, then return the result of standard setattr setting. I've added a test for some of these behaviors. -Travis ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
On Sunday 29 October 2006 02:40, Travis Oliphant wrote:
I've committed some fixes to this particular issue that allows setattr to be used to set field names.
I just started playing with the new recarray: that works great ! Travis, Thanks a lot ! ------------------------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
participants (3)
-
Michael McNeil Forbes
-
Pierre GM
-
Travis Oliphant