comparing string array with None raises ValueError
Hi, I'm not sure if this is fixed already in CVS but here it goes: I'm working with record arrays, and trying to access a field with type '10a80' - that is, an array of 10 80 char 'strings' :
q.Mrc.hdr = q.Mrc.hdrArray[0].field q.Mrc.hdr('title') != None Traceback (most recent call last): File "<input>", line 1, in ? File "/jws30/haase/PrLin/numarray/strings.py", line 431, in __ne__ return self.StrCmp(other).__ne__(0) File "/jws30/haase/PrLin/numarray/strings.py", line 385, in StrCmp b = asarray(b0, kind=self.__class__) File "/jws30/haase/PrLin/numarray/strings.py", line 1024, in asarray return array(buffer, itemsize, shape, byteoffset, bytestride, kind) File "/jws30/haase/PrLin/numarray/strings.py", line 994, in array byteoffset=byteoffset, bytestride=bytestride) File "/jws30/haase/PrLin/numarray/strings.py", line 84, in __init__ raise ValueError("Must define both shape & itemsize if buffer is None") ValueError: Must define both shape & itemsize if buffer is None
print q.Mrc.hdr('title') ['tit1e seb says hi' '' '' '' '' '' '' '' '' '']
Thanks for numarray ;-) Sebastian Haase
On Thu, 2004-06-24 at 13:06, Sebastian Haase wrote:
Hi, I'm not sure if this is fixed already in CVS but here it goes: I'm working with record arrays, and trying to access a field with type '10a80' - that is, an array of 10 80 char 'strings' :
q.Mrc.hdr = q.Mrc.hdrArray[0].field q.Mrc.hdr('title') != None
Shouldn't this be: q.Mrc.hdr('title') != "" Regards, Todd
On Thursday 24 June 2004 10:31 am, Todd Miller wrote:
On Thu, 2004-06-24 at 13:06, Sebastian Haase wrote:
Hi, I'm not sure if this is fixed already in CVS but here it goes: I'm working with record arrays, and trying to access a field with
type '10a80' - that is, an array of 10 80 char 'strings' :
q.Mrc.hdr = q.Mrc.hdrArray[0].field q.Mrc.hdr('title') != None
Shouldn't this be:
q.Mrc.hdr('title') != ""
No, I understand that this makes more sense, but I have some "display hook"-code that compares everything with None... In general it must be OK to compare anything with None, right ? (BTW, I get the same error with == and !=) Regards, Sebastian Haase
On Thursday 24 June 2004 10:31 am, Todd Miller wrote:
On Thu, 2004-06-24 at 13:06, Sebastian Haase wrote:
Hi, I'm not sure if this is fixed already in CVS but here it goes: I'm working with record arrays, and trying to access a field with
type '10a80' - that is, an array of 10 80 char 'strings' :
q.Mrc.hdr = q.Mrc.hdrArray[0].field q.Mrc.hdr('title') != None
Shouldn't this be:
q.Mrc.hdr('title') != ""
(in my first reply, I forgot to point out that q.Mrc.hdr('title') is an ARRAY of strings ! ) No, I understand that this makes more sense, but I have some "display hook"-code that compares everything with None... In general it must be OK to compare anything with None, right ? (BTW, I get the same error with == and !=) Regards, Sebastian Haase
On Thu, 2004-06-24 at 13:46, Sebastian Haase wrote:
On Thursday 24 June 2004 10:31 am, Todd Miller wrote:
On Thu, 2004-06-24 at 13:06, Sebastian Haase wrote:
Hi, I'm not sure if this is fixed already in CVS but here it goes: I'm working with record arrays, and trying to access a field with
type '10a80' - that is, an array of 10 80 char 'strings' :
q.Mrc.hdr = q.Mrc.hdrArray[0].field q.Mrc.hdr('title') != None
Shouldn't this be:
q.Mrc.hdr('title') != ""
(in my first reply, I forgot to point out that q.Mrc.hdr('title') is an ARRAY of strings ! )
No, I understand that this makes more sense, but I have some "display hook"-code that compares everything with None... In general it must be OK to compare anything with None, right ? (BTW, I get the same error with == and !=)
OK, I see your point. I talked it over with Perry and he made a reasonable case for allowing comparisons with None (or any object). Perry argued that since None is a common default parameter value, it might simplify code to not have to add logic to handle that case. If no one objects, I'll change numarray.strings so that comparison of a string array with any object not convertible to a string array results in an array of False values. Any objections? Regards, Todd
On 24 Jun 2004, Todd Miller wrote:
OK, I see your point. I talked it over with Perry and he made a reasonable case for allowing comparisons with None (or any object). Perry argued that since None is a common default parameter value, it might simplify code to not have to add logic to handle that case.
If no one objects, I'll change numarray.strings so that comparison of a string array with any object not convertible to a string array results in an array of False values.
Any objections?
Hmm, before you do that you might look at this: >>> import numarray >>> b = numarray.zeros(10) >>> b==0 array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1], type=Bool) >>> b==None 0 So comparing numeric arrays to None returns a scalar false value since the variable b is not None. I figure that the behavior of comparisons to None for string arrays and numeric arrays ought to be consistent. I don't know which is preferred... Rick
On Thursday 24 June 2004 11:46 am, Todd Miller wrote:
On Thu, 2004-06-24 at 13:46, Sebastian Haase wrote:
On Thursday 24 June 2004 10:31 am, Todd Miller wrote:
On Thu, 2004-06-24 at 13:06, Sebastian Haase wrote:
Hi, I'm not sure if this is fixed already in CVS but here it goes: I'm working with record arrays, and trying to access a field with
type '10a80' - that is, an array of 10 80 char 'strings' :
> q.Mrc.hdr = q.Mrc.hdrArray[0].field > q.Mrc.hdr('title') != None
Shouldn't this be:
q.Mrc.hdr('title') != ""
(in my first reply, I forgot to point out that q.Mrc.hdr('title') is an ARRAY of strings ! )
No, I understand that this makes more sense, but I have some "display hook"-code that compares everything with None... In general it must be OK to compare anything with None, right ? (BTW, I get the same error with == and !=)
OK, I see your point. I talked it over with Perry and he made a reasonable case for allowing comparisons with None (or any object). Perry argued that since None is a common default parameter value, it might simplify code to not have to add logic to handle that case.
If no one objects, I'll change numarray.strings so that comparison of a string array with any object not convertible to a string array results in an array of False values.
Any objections?
Just my feeling is, that a single None would do it ... Regards, Sebastian
On Thu, 2004-06-24 at 14:53, Rick White wrote:
On 24 Jun 2004, Todd Miller wrote:
OK, I see your point. I talked it over with Perry and he made a reasonable case for allowing comparisons with None (or any object). Perry argued that since None is a common default parameter value, it might simplify code to not have to add logic to handle that case.
If no one objects, I'll change numarray.strings so that comparison of a string array with any object not convertible to a string array results in an array of False values.
Any objections?
Hmm, before you do that you might look at this:
>>> import numarray >>> b = numarray.zeros(10) >>> b==0 array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1], type=Bool) >>> b==None 0
So comparing numeric arrays to None returns a scalar false value since the variable b is not None.
Oh yeah... I forgot about that. Numeric comparisons have a special case for None and blow up for most other arbitrary objects. So there's also:
a= numarray.arange(100) a == None 0 class foo: .. pass .. a == foo Traceback (most recent call last): .. TypeError: UFunc arguments must be numarray, scalars or numeric sequences
I figure that the behavior of comparisons to None for string arrays and numeric arrays ought to be consistent. I don't know which is preferred...
So the two choices now on the table are: 1. Change string equality comparisons to return an array of false or true for objects which don't convert to strings. Change Numeric equality comparisons in a similar fashion. 2. Special case string equality comparisons for None, as is done with numerical comparisons. Raise a type error for objects (other than None) which don't convert to string arrays. I think I like 2. Other opinions on 1 & 2? Other choices? Regards, Todd
On June 24, 2004 01:46 pm, Sebastian Haase wrote:
In general it must be OK to compare anything with None, right ? (BTW, I get the same error with == and !=)
No! Not in general! I learnt this back when Numeric implemented rich comparisions; suddenly, lots of my code broke. You don't actually want "is this object _equal_ (or not equal) to None", you want "is this object None", as None is a singleton. A contrived example:
class A: ... def __eq__(self, other): return True ... a = A()
So, this is bad:
a == None True
This is good:
a is None False
In general, if you're testing if something is None, use 'is', not '=='.
q.Mrc.hdr('title') is not None
-- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca
Todd Miller wrote:
So the two choices now on the table are:
1. Change string equality comparisons to return an array of false or true for objects which don't convert to strings. Change Numeric equality comparisons in a similar fashion.
2. Special case string equality comparisons for None, as is done with numerical comparisons. Raise a type error for objects (other than None) which don't convert to string arrays.
I think I like 2. Other opinions on 1 & 2? Other choices?
Regards, Todd
It would seem consistency with numerical arrays is important so unless someone presents a compelling argument for changing that behavior for arrays, the choice appears clear. Perry [wishing he had spent a little more time thinking about it and trying to remember why None is special cased for comparisons]
On Thu, 2004-06-24 at 15:26, David M. Cooke wrote:
On June 24, 2004 01:46 pm, Sebastian Haase wrote:
In general it must be OK to compare anything with None, right ? (BTW, I get the same error with == and !=)
No! Not in general!
Well, this is a good point. I think the current numerical behavior was a hack I stuck in for people who might not be aware of "is". It's looking like a mistake now.
I learnt this back when Numeric implemented rich comparisions; suddenly, lots of my code broke. You don't actually want "is this object _equal_ (or not equal) to None", you want "is this object None", as None is a singleton.
However, given the context of the original question, Sebastian's code doesn't read like *that* kind of None comparison:
type '10a80' - that is, an array of 10 80 char 'strings' :
> q.Mrc.hdr = q.Mrc.hdrArray[0].field > q.Mrc.hdr('title') != None
q.Mrc.hdr('title') is pretty clearly a character array, ergo, it's not None. What did you want it to do Sebastian? Regards, Todd
On Thursday 24 June 2004 01:10 pm, Todd Miller wrote:
On Thu, 2004-06-24 at 15:26, David M. Cooke wrote:
On June 24, 2004 01:46 pm, Sebastian Haase wrote:
In general it must be OK to compare anything with None, right ? (BTW, I get the same error with == and !=)
No! Not in general!
Well, this is a good point. I think the current numerical behavior was a hack I stuck in for people who might not be aware of "is". It's looking like a mistake now.
I learnt this back when Numeric implemented rich comparisions; suddenly, lots of my code broke. You don't actually want "is this object _equal_ (or not equal) to None", you want "is this object None", as None is a singleton.
However, given the context of the original question, Sebastian's code
doesn't read like *that* kind of None comparison:
type '10a80' - that is, an array of 10 80 char 'strings' :
>> q.Mrc.hdr = q.Mrc.hdrArray[0].field >> q.Mrc.hdr('title') != None
q.Mrc.hdr('title') is pretty clearly a character array, ergo, it's not None. What did you want it to do Sebastian?
q.Mrc.hdr('title') is rather an array of 10 'character array' .. that's different, isn't it ? In any case, I actually already changed my code to the check of 'is None' - which is probably what I wanted anyway. So, the only argument left, is that the original error/exception message I got was kind of "ugly" ;-) And I think/thought if 'something' cannot be _converted_ to a string array it can't _be_ '==' a string array. That's just why I expected the simple result to be 'False'. But now I understand that if comparing an array to a scalar (maybe 'None') you always have to decide if this actually compares "the whole array" with the scalar, or if the comparison get "put through" and done on each element "separately" ... which it seems is already decided at least in the numerical array case. Regards, Sebastian
On Thu, 2004-06-24 at 16:38, Sebastian Haase wrote:
On Thursday 24 June 2004 01:10 pm, Todd Miller wrote:
On Thu, 2004-06-24 at 15:26, David M. Cooke wrote:
On June 24, 2004 01:46 pm, Sebastian Haase wrote:
In general it must be OK to compare anything with None, right ? (BTW, I get the same error with == and !=)
No! Not in general!
Well, this is a good point. I think the current numerical behavior was a hack I stuck in for people who might not be aware of "is". It's looking like a mistake now.
I learnt this back when Numeric implemented rich comparisions; suddenly, lots of my code broke. You don't actually want "is this object _equal_ (or not equal) to None", you want "is this object None", as None is a singleton.
However, given the context of the original question, Sebastian's code
doesn't read like *that* kind of None comparison:
type '10a80' - that is, an array of 10 80 char 'strings' : >>> q.Mrc.hdr = q.Mrc.hdrArray[0].field >>> q.Mrc.hdr('title') != None
q.Mrc.hdr('title') is pretty clearly a character array, ergo, it's not None. What did you want it to do Sebastian?
q.Mrc.hdr('title') is rather an array of 10 'character array'
I thought it was a single nrows x 10 CharArray with 80 character elements.
.. that's different, isn't it ?
It would be. What does q.Mrc.hdr('title').info() say?
In any case, I actually already changed my code to the check of 'is None' - which is probably what I wanted anyway. So, the only argument left, is that the original error/exception message I got was kind of "ugly" ;-) And I think/thought if 'something' cannot be _converted_ to a string array it can't _be_ '==' a string array. That's just why I expected the simple result to be 'False'. But now I understand that if comparing an array to a scalar (maybe 'None') you always have to decide if this actually compares "the whole array" with the scalar, or if the comparison get "put through" and done on each element "separately" ... which it seems is already decided at least in the numerical array case.
I'm not really sure that the numerical array None hack should be replicated. Barring an edict from Perry or Rick or an outcry from the community, I think I'm going to just leave this alone. Regards, Todd
On June 24, 2004 04:10 pm, Todd Miller wrote:
On Thu, 2004-06-24 at 15:26, David M. Cooke wrote:
On June 24, 2004 01:46 pm, Sebastian Haase wrote:
In general it must be OK to compare anything with None, right ? (BTW, I get the same error with == and !=)
No! Not in general!
Well, this is a good point. I think the current numerical behavior was a hack I stuck in for people who might not be aware of "is". It's looking like a mistake now.
I've fixed up all uses of == None and != None in my copy of the numarray CVS, but the anonymous CVS on SourceForge is timing out :-(, so I haven't made a patch yet. Basically, I ran 'grep -R "[!=]= *None" *' to find all uses of == and != None, and replaced them with is/is not None. That would remove possible bugs. -- |>|\/|< /--------------------------------------------------------------------------\ |David M. Cooke http://arbutus.physics.mcmaster.ca/dmc/ |cookedm@physics.mcmaster.ca
participants (5)
-
David M. Cooke -
Perry Greenfield -
Rick White -
Sebastian Haase -
Todd Miller