Alan G Isaac wrote:

On Mon, 31 Oct 2005, Robert Cimrman apparently wrote:

...

is this the expected behaviour? IMHO (b * c) == (b and c), (b + c) == (b or c) should hold...

I expected the Boolean operations to yield element-by-element comparisons. What are they?? In contrast, the + and * operators give the expected results.

This is a Python deal. It would be nice if b and c did the same thing as b * c, but Python does not allow overloading of the "and" and "or" operators (A PEP to say it should would be possible). Thus, "b and c" evaluates the truth of b and the truth of c as a whole (not elementwise), and there is no way to over-ride this. -Travis

David M. Cooke wrote:

On Oct 31, 2005, at 07:37, Alan G Isaac wrote:

...

On Mon, 31 Oct 2005, Robert Cimrman apparently wrote:

...

is this the expected behaviour? IMHO (b * c) == (b and c), (b + c) == (b or c) should hold...

I expected the Boolean operations to yield element-by-element comparisons. What are they?? In contrast, the + and * operators give the expected results.

Use & and | instead of 'and' and 'or'.

Thanks, this works as expected. Is there a place where such things are documented? I suspect many people would expect 'and' and '&' to have the same behaviour. Of course, looking at the numeric operations special methods should reveal the problem, but who would do it ;) r.

On Tue, 1 Nov 2005, Ed Schofield apparently wrote:

Hmmm. Could we somehow get "b and c" to return a simple boolean rather than an array of booleans? This would be more consistent. At the moment it's totally mad: "b and c" is different to "c and b" ;)

You're just getting back c in the first case and b in the second case, as "expected" (once reminded of this Python behavior). You can look at bool(b and c) and get the right result. fwiw, Alan Isaac

The recent question about use of "and" and "or" highlighted an issue I think is worth at least a little discussion (I don't recall it being discussed for scipy_core, but maybe it already has been). numarray doesn't permit using arrays as truth values since we figured that it wasn't very clear what people expected to happen (this case is a good illustration). I'd like to make the argument that scipy_core also not permit arrays to be used as truth values (i.e., one should get an exception if one tries to use it that way). I realize that this will break some old code, but since incompatible changes are being made, this is the time to make this sort of change. If left in, it is going to bite people, often quietly. Perry

Perry Greenfield wrote:

The recent question about use of "and" and "or" highlighted an issue I think is worth at least a little discussion (I don't recall it being discussed for scipy_core, but maybe it already has been).

It's a good discussion to have. I don't recall talking about it.

numarray doesn't permit using arrays as truth values since we figured that it wasn't very clear what people expected to happen (this case is a good illustration). I'd like to make the argument that scipy_core also not permit arrays to be used as truth values (i.e., one should get an exception if one tries to use it that way). I realize that this will break some old code, but since incompatible changes are being made, this is the time to make this sort of change. If left in, it is going to bite people, often quietly.

I agree it can bite people, but I'm concerned that arrays not having a truth value is an odd thing in Python --- you have to implement it by raising an error when __nonzero__ is called right? All other objects in Python have truth values (including its built-in array). My attitude is that its just better to teach people the proper use of truth values, then to break form with the rest of Python. I'm would definitely like to hear more opinions though. It would be very easy to simply raise and error when __nonzero__ is called. -Travis

Robert Cimrman wrote:

...

...

numarray doesn't permit using arrays as truth values since we figured that it wasn't very clear what people expected to happen (this case is a good illustration). I'd like to make the argument that scipy_core also not permit arrays to be used as truth values (i.e., one should get an exception if one tries to use it that way). I realize that this will break some old code, but since incompatible changes are being made, this is the time to make this sort of change. If left in, it is going to bite people, often quietly.

I agree it can bite people, but I'm concerned that arrays not having a truth value is an odd thing in Python --- you have to implement it by raising an error when __nonzero__ is called right?

All other objects in Python have truth values (including its built-in array). My attitude is that its just better to teach people the proper use of truth values, then to break form with the rest of Python.

I'm would definitely like to hear more opinions though. It would be very easy to simply raise and error when __nonzero__ is called.

Speaking about 'and' only, my problem with the current implementation of it is that it _looks_ like working as '*' in some cases - 'b and c' returns an array whose length is that of b and c (if the lengths are equal, that is). I would not be against 'b and c' giving a single True or False... But this also breaks the Python semantics of 'and'. The same holds for other logical ops, of course.

So I don't know :-) - I can live with the current state.

I'm slightly in favour of raising an exception like in nummaray. But I'd like to point out an inconsistency between the current truth values of ndarrays and those of other Python objects:

...

...

l = [False, False] print bool(l) s = set(l) print bool(s) import array a = array.array('b',l) print bool(a) import scipy nd1 = scipy.array(l, 'b') nd2 = scipy.array(l, '?') print bool(nd1) print bool(nd2)

gives: True True True False False If we do adopt Python's strange idiom with logical operators we should probably make arrays' truth values consistent with Python objects too. -- Ed

On Nov 7, 2005, at 8:20 AM, Steven H. Rogers wrote:

Concur that allowing arrays as truth values is the pythonic thing to do. Please do it as long as it's documented.

Regards, Steve

Travis Oliphant wrote:

...

...

I agree it can bite people, but I'm concerned that arrays not having a truth value is an odd thing in Python --- you have to implement it by raising an error when __nonzero__ is called right?

All other objects in Python have truth values (including its built-in array). My attitude is that its just better to teach people the proper use of truth values, then to break form with the rest of Python.

I'm would definitely like to hear more opinions though. It would be very easy to simply raise and error when __nonzero__ is called.

-Travis

I guess I'd like to challenge the pythonicity of always having truth values. The situation here may be unique (so far, I'm not aware of any other similar cases). When Python allowed rich comparisons, it meant such comparisons didn't have to return simple boolean values. It was implemented almost entirely to satisfy Numeric users. This lead to many users believing that the results of such comparisons could now be used with "and" and "or" since as far as they were concerned, the results did result in booleans (arrays, of course). My argument is that because of this mismatch we can be *sure* that many users will try to use arrays this way, regardless of how many warnings you put in the documentation. And the vast majority of the time, they will not get what they intended. And for many of these cases, that there was a mistake may not surface right away (they are going to get as a result an array, usually of the right shape, if not the right type). For me this is a very big drawback. Let's look at the other side. Is it obvious what arrays are false? The pythonic thing would to make them false if they could be empty. And sure, they can be. But that is a pretty rare case of usage. Who is going to use that case very often? Besides, with rank-0 values, then people would think the pythonic thing is to treat them as false, but they aren't empty. But some might think they should be false if all values are 0 (the current Numeric behavior). This case would at least be used practically, but it is at odds with how lists and other sequences behave. Essentially, my argument is that this case presents many likely surprises for users and when there isn't one obvious way to do it, it shouldn't be done. I'd argue that not allowing arrays as truth values it is more Pythonic. (Existing arrays don't support rich comparisons, and they are false only when empty, as would be expected). Does anyone else have examples that use rich comparisons to produce sequences in other libraries that are treated as truth values? Perry

OK. I can't think of a really good use case for using an array as a truth value. I would argue though, that it would make sense for an array of zeros to be False and an array with any non-zero values to be True. Travis Oliphant wrote:

Perry Greenfield wrote:

...

Essentially, my argument is that this case presents many likely surprises for users and when there isn't one obvious way to do it, it shouldn't be done. I'd argue that not allowing arrays as truth values it is more Pythonic. (Existing arrays don't support rich comparisons, and they are false only when empty, as would be expected).

Right now, I'm leaning towards raising an error. A big explanation for that is the idea that "explicit" is better than "implicit." I'm persuaded by Perry's argument that a casual user is going to think that "and" and "&" have the same behavior for arrays. The experienced user is going to understand the difference, anyway.

So ultimately, I don't really see an impressive use case for returning a truth value for arrays, but I do see a "new-user" issue if an error is not raised.

So, right now in SVN, arrays as truth values raise erros unless the array contains only one element (in which case it is unambiguous).

-Travis

_______________________________________________ Scipy-dev mailing list Scipy-dev@scipy.net http://www.scipy.net/mailman/listinfo/scipy-dev

-- Steven H. Rogers, Ph.D., steve@shrogers.com Weblog: http://shrogers.com/weblog "He who refuses to do arithmetic is doomed to talk nonsense." -- John McCarthy

On Mon, 07 Nov 2005, Travis Oliphant apparently wrote:

There will be many though, I'm afraid, who think b or a ought to return element-wise like b | a does. This is not possible in Python. Raising an error will at least alert them to the problem which might otherwise give them misleading results.

User perspective: I find this persuasive, even though I am unhappy that it means a standard Python behavior will disappear. What's more, if the SciPy community should ultimately change its mind, it looks easy to back out of raising an error while hard to back out of accepting the standard behavior. So it seems the right way to go now as long as the ultimate outcome is in doubt. Cheers, Alan Isaac

Ed Schofield wrote:

I agree with this reasoning, but I'd like to illustrate a drawback to the new behaviour:

...

...

...

a1 = array([1,2,3]) a2 = a1 if a1 == a2:

... print "equal" ... Traceback (most recent call last): File "<stdin>", line 1, in ? ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

It's not a drawback. This is exactly the reason we're making the change.

Using == and != to compare arrays was simple and (I think) unambiguous before. It would be nice to allow these comparisons again, while raising an exception for the general case. Perhaps we could modify arrays' __eq__ and __neq__ methods to call .any() and .all() for us, returning a single truth value, rather than returning an array of truth values as it does currently?

No. Rich comparisons were added to the language precisely *for Numeric* so we could return arrays instead of just True or False. We're not going to regress to the Python 2.0 era. In any case, you can't have those methods "call .any() and .all() for us;" there really is an ambiguity. We don't know beforehand which one you want called. And in the face of ambiguity, we're refusing the temptation to guess.

We still have scipy.equal() and scipy.not_equal() for elementwise comparisons.

This might actually cause less code breakage (like for me ;), since using == and != in conditional expressions would work as before.

I think you may need to reexamine your code that uses that construction. The old behavior was implicitly the same as any(a1 == a2) not all(a1 == a2). It's likely that you wanted the latter not the former.

It would also have the bonus of bringing SciPy's behaviour closer to that of Python's builtin objects and existing 1-d array module:

Any API compatibility with the stdlib's array module is entirely coincidental. It's not a goal and never will be. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter

On Nov 8, 2005, at 9:21 AM, Ed Schofield wrote:

I agree with this reasoning, but I'd like to illustrate a drawback to the new behaviour:

...

...

...

a1 = array([1,2,3]) a2 = a1 if a1 == a2: ... print "equal" ...

What about:

...

...

a1 = array([[1,2,3],[1,2,3]]) a2 = array([1,2,3]) a1 == a2

These two arrays are not the same shape but because of broadcasting will show to be equal. Is this what you intended? Some might, some might not. Robert has already pointed out that lots of people want == to result in an array of booleans (most I'd argue) rather than a single boolean value. And if you wanted to use the current Numeric behavior, then

...

...

array([0,0]) == array([0,1])

will not do what you wish it since there is at least one equal element, it is treated as true. (again reiterating Robert's point.) Your example illustrates exactly why allowing this behavior is dangerous. Two different people looking at this may expect two different results. Perry

Ed Schofield wrote:

On Tue, 8 Nov 2005, Robert Kern wrote:

...

In any case, you can't have those methods "call .any() and .all() for us;" there really is an ambiguity. We don't know beforehand which one you want called. And in the face of ambiguity, we're refusing the temptation to guess.

We've seen that there's an ambiguity in the case of logical operations on arrays like "a and b", "a or b". But I don't see any ambiguity in the case of == and !=. Can two arrays be considered 'equal' if any of their elements differ? ;)

== as an operation doesn't test the equality of the arrays as a whole. It returns an array with the results of the == comparison element-by-element. We don't want it to return a single truth value. If you need more semantics on top of that, then use .any() or .all() or whatever else. We lobbied extensively for Python to allow ==, !=, <, >, etc. to be able to return non-Boolean values. We got that capability in Python 2.1. We're not going to change that decision now years after the fact. If you have a concern about code breakage, regressing now would break a huge amount of code, much more than disabling .__nonzero__().

You're right that my code that now raises an exception was silently wrong before. That's a definite step forward.

I hoped that would have been dispositive.

...

Any API compatibility with the stdlib's array module is entirely coincidental. It's not a goal and never will be.

Yes, it is a goal. The recent conversion of typecodes for better compatibility with 'array' and 'struct' is an example. When there's no good argument to differ we might as well be consistent with the standard library.

Numeric exists because the API of stdlib's array module was inadequate for our purposes. We've rationalized the type characters primarily to match Python as a whole, not array specifically. Conventions about data are a different thing than object behavior. -- Robert Kern rkern@ucsd.edu "In the fields of hell where the grass grows high Are the graves of dreams allowed to die." -- Richard Harter

Ryan Gutenkunst wrote:

On Nov 8, 2005, at 10:35 AM, Perry Greenfield wrote:

...

Robert has already pointed out that lots of people want == to result in an array of booleans (most I'd argue) rather than a single boolean value.

I'm coming late to this discussion, but I'd like to mention that a similar issue with the old scipy burned me just today.

Guido's PEP 8 Style Guide suggests: - For sequences, (strings, lists, tuples), use the fact that empty sequences are false, so "if not seq" or "if seq" is preferable to "if len(seq)" or "if not len(seq)".

However, an old scipy array containing any number of zeros is 'False', as illustrated below. (I haven't tried this on new scipy.)

...

...

...

scipy.__version__ '0.3.3_303.4601' if [0]: ... print 'hello' ... hello if scipy.array([0, 0]): ... print 'hello' ...

So, do we want empty arrays to return false and not return an error? This is definitely a possibility. -Travis

On 11/7/05, Travis Oliphant <oliphant@ee.byu.edu> wrote:

Steven H. Rogers wrote:

...

OK. I can't think of a really good use case for using an array as a truth value. I would argue though, that it would make sense for an array of zeros to be False and an array with any non-zero values to be True.

I agree this makes sense. That's why it used to be the default behavior. But you can already get that behavior with any(a).

There will be many though, I'm afraid, who think b or a ought to return element-wise like b | a does. This is not possible in Python. Raising an error will at least alert them to the problem which might otherwise give them misleading results.

I'm not quite sure how any() is supposed to work; does it just return true if one or more element evaluates to true? In my current code, I have the following: if sum(lower>=median or median>=upper): <do something> which returns a ValueError. What is the best way to detect elements in one array that are less than the corresponding element in the other without constructing a list comprehension? Thanks for the clarification, -- Chris Fonnesbeck Atlanta, GA

I'm not quite sure how any() is supposed to work; does it just return true if one or more element evaluates to true?

Yes. Exactly. There is an axis argument, but for just testing truth of anything in the array you don't want to use it.

In my current code, I have the following:

if sum(lower>=median or median>=upper): <do something>

which returns a ValueError. What is the best way to detect elements in one array that are less than the corresponding element in the other without constructing a list comprehension?

The ValueError was just recently added because it is ambiguous as to what you meant by this. I would say if any(lower>=media) or any(median>=upper): <do something> -Travis