Enum, Flag, __contains__, and False vs TypeError
API question. Background: ---------- When doing checks such as --> 3 in [4, 5, 6] --> 'test' in {'test':True, 'live':False} the result is True or False. When doing checks such as --> 3 in 'hello world' --> [4, 5, 6] in {'test':True, 'live':False} the result is a TypeError. The general rule seems to be that if it is impossible for the in-question object to be in the container object then a TypeError is raised, otherwise True or False is returned. Question 1: ---------- (A) A standard Enum class is a container of Enum members. It cannot hold anything else. However, it has been returning False both in cases where the in-question object was an Enum of a different class (a Fruit in a Color, for example) and when the in-question object was not even an Enum ('apple' in Fruit, for example). (B) The waters get even more muddied when Fruit has a str mixin, so `Fruit.APPLE == 'apple' is True` -- in that case, should `'orange' in Fruit` return True, False, or raise TypeError? Question 2: ---------- (A) The new Flag type allows `in` tests on the members themselves; so, for example: --> SomeFlag.ONE in SomeFlag.One|SomeFlag.TWO True The question, of course, is what to do when a non-Flag member is tested for: --> 'apple' in SomeFlag.ONE # False or TypeError? --> 2 in SomeFlag.TWO # True or TypeError? (B) And, of course, the same muddier question arises with IntFlag, where SomeFlag.TWO == 2 is True. My thoughts: ----------- For question 1A (pure Enum): I'm thinking a TypeError should be raised when the in-question object is not an Enum member of any kind -- it simply is not possible for that object to ever be in an Enum, and is surely a bug. For question 1B (mixed Enum): if 1A is TypeError, then 1B should also be TypeError at least for non-mixin in-question types (so checking for 1 in StrEnum would be a TypeError), but I'm torn between TypeError and True/False for cases where the in-question type matches the mixin type ('apple' in StrEnum).... On the one hand, even though an Enum member might be equal to some other type, that other type will not have the Enum attributes, etc, and a True answer would lead one to believe you could access `.name` and `.value`, etc., while a False answer would lead one to believe there was no match even though equality tests pass; on the other hand, how strong is the "container" aspect of a mixed Enum? How often is the test `'apple' in Fruit` meant to discover if you have a Fruit member vs whether you have something that could be a Fruit member? Also, how important is it to be consistent with IntFlag, which I definitely think should return True/False for int checks? For question 2A (pure Flag): I'm comfortable sticking with a TypeError (assuming we switch to TypeError for 1A). For question 2B (int Flag): I think a TypeError if the in-question object is not an int is appropriate (assuming TypeError for 1A and 2A), but if it is an int, True/False seems the better path. My reasoning being that Flag and IntFlag are more similar to sets than lists, and IntFlag is specifically meant to work with ints, and a test of `2 in some_int_flags` is more concerned with a flag being set than with .name or .value attributes that may or may not exist on the in-question object. Any and all thoughts appreciated. -- ~Ethan~
On Wed, 4 Apr 2018 at 11:30 Ethan Furman
API question.
Background: ----------
When doing checks such as
--> 3 in [4, 5, 6] --> 'test' in {'test':True, 'live':False}
the result is True or False.
When doing checks such as
--> 3 in 'hello world' --> [4, 5, 6] in {'test':True, 'live':False}
the result is a TypeError.
The general rule seems to be that if it is impossible for the in-question object to be in the container object then a TypeError is raised, otherwise True or False is returned.
Question 1: ----------
(A) A standard Enum class is a container of Enum members. It cannot hold anything else. However, it has been returning False both in cases where the in-question object was an Enum of a different class (a Fruit in a Color, for example) and when the in-question object was not even an Enum ('apple' in Fruit, for example).
(B) The waters get even more muddied when Fruit has a str mixin, so `Fruit.APPLE == 'apple' is True` -- in that case, should `'orange' in Fruit` return True, False, or raise TypeError?
Question 2: ----------
(A) The new Flag type allows `in` tests on the members themselves; so, for example:
--> SomeFlag.ONE in SomeFlag.One|SomeFlag.TWO True
The question, of course, is what to do when a non-Flag member is tested for:
--> 'apple' in SomeFlag.ONE # False or TypeError?
--> 2 in SomeFlag.TWO # True or TypeError?
(B) And, of course, the same muddier question arises with IntFlag, where SomeFlag.TWO == 2 is True.
My thoughts: -----------
For question 1A (pure Enum): I'm thinking a TypeError should be raised when the in-question object is not an Enum member of any kind -- it simply is not possible for that object to ever be in an Enum, and is surely a bug.
Yes, although it sounds like it already does the other thing, so do you want to start raising TypeError immediately or go for a deprecation period?
For question 1B (mixed Enum): if 1A is TypeError, then 1B should also be TypeError at least for non-mixin in-question types (so checking for 1 in StrEnum would be a TypeError), but I'm torn between TypeError and True/False for cases where the in-question type matches the mixin type ('apple' in StrEnum).... On the one hand, even though an Enum member might be equal to some other type, that other type will not have the Enum attributes, etc, and a True answer would lead one to believe you could access `.name` and `.value`, etc., while a False answer would lead one to believe there was no match even though equality tests pass; on the other hand, how strong is the "container" aspect of a mixed Enum? How often is the test `'apple' in Fruit` meant to discover if you have a Fruit member vs whether you have something that could be a Fruit member? Also, how important is it to be consistent with IntFlag, which I definitely think should return True/False for int checks?
I think this is a design question as to how interchangeable you want these kind of enums to be, and I personally have no opinion as I have never needed to use them.
For question 2A (pure Flag): I'm comfortable sticking with a TypeError (assuming we switch to TypeError for 1A).
Makes sense.
For question 2B (int Flag): I think a TypeError if the in-question object is not an int is appropriate (assuming TypeError for 1A and 2A), but if it is an int, True/False seems the better path. My reasoning being that Flag and IntFlag are more similar to sets than lists, and IntFlag is specifically meant to work with ints, and a test of `2 in some_int_flags` is more concerned with a flag being set than with .name or .value attributes that may or may not exist on the in-question object.
Same as above; no experience so no opinion.
Any and all thoughts appreciated.
-- ~Ethan~ _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/brett%40python.org
On Apr 4, 2018, at 11:32, Ethan Furman
(A) A standard Enum class is a container of Enum members. It cannot hold anything else. However, it has been returning False both in cases where the in-question object was an Enum of a different class (a Fruit in a Color, for example) and when the in-question object was not even an Enum ('apple' in Fruit, for example).
(B) The waters get even more muddied when Fruit has a str mixin, so `Fruit.APPLE == 'apple' is True` -- in that case, should `'orange' in Fruit` return True, False, or raise TypeError?
Are you proposing to change current behavior, and if so, what’s your deprecation plan? I’m not sure I feel that the purity is important enough to change how it currently works, especially since you’ll have to be prepared to catch exceptions rather than just handle the boolean value. OTOH, since most of my use cases are comparisons against explicit enum values, I’m not sure how often people write code to check for enum values contained in the Enum (I do it in one or two places where I’m deserializing the actual value object, e.g. from a pickle). (FWIW, I encourage individual comparisons use `is` rather than `==`.)
Question 2: ----------
(A) The new Flag type allows `in` tests on the members themselves; so, for example:
--> SomeFlag.ONE in SomeFlag.One|SomeFlag.TWO True
The question, of course, is what to do when a non-Flag member is tested for:
--> 'apple' in SomeFlag.ONE # False or TypeError?
--> 2 in SomeFlag.TWO # True or TypeError?
(B) And, of course, the same muddier question arises with IntFlag, where SomeFlag.TWO == 2 is True.
Well, now I’m confused: Python 3.7.0b2+ (heads/3.7:f328caf4ca, Mar 26 2018, 19:57:33) [Clang 9.0.0 (clang-900.0.39.2)] on darwin Type "help", "copyright", "credits" or "license" for more information.
from enum import IntFlag class Flag(IntFlag): ... a = 1 ... b = 2 ... c = 4 ... d = 8 ... 'foo' in (Flag.a|Flag.b) True
Cheers, -Barry
On 04/04/2018 12:40 PM, Barry Warsaw wrote:
On Apr 4, 2018, at 11:32, Ethan Furman wrote:
(A) A standard Enum class is a container of Enum members. It cannot hold anything else...
(B) The waters get even more muddied when Fruit has a str mixin, so `Fruit.APPLE == 'apple' is True`...
Are you proposing to change current behavior, and if so, what’s your deprecation plan?
The first problem is to determine what the correct behavior should be -- if that turns out to be different than what is we can then decide the time frame for changing it. ;)
I’m not sure I feel that the purity is important enough to change how it currently works, especially since you’ll have to be prepared to catch exceptions rather than just handle the boolean value.
It's not so much about purity as about bugs not passing silently. But whether or not it's a bug depends on how "containerish" an Enum class is (or is perceived to be).
OTOH, since most of my use cases are comparisons against explicit enum values, I’m not sure how often people write code to check for enum values contained in the Enum (I do it in one or two places where I’m deserializing the actual value object, e.g. from a pickle).
I write a bunch of cli scripts for work, so I'm often comparing a string value with an Enum value. As often as not, I don't need the member itself, just to know that the string I have can be converted to a valid Enum -- `if "quick" in Method`, for example -- but just converting to an Enum is undoubtedly the better course of action .
(FWIW, I encourage individual comparisons use `is` rather than `==`.)
Which is great for pure Enums, not so much for mixed.
Question 2: ----------
(A) The new Flag type allows `in` tests on the members themselves; so, for example:
--> SomeFlag.ONE in SomeFlag.One|SomeFlag.TWO True
The question, of course, is what to do when a non-Flag member is tested for:
--> 'apple' in SomeFlag.ONE # False or TypeError?
--> 2 in SomeFlag.TWO # True or TypeError?
(B) And, of course, the same muddier question arises with IntFlag, where SomeFlag.TWO == 2 is True.
Well, now I’m confused:
Python 3.7.0b2+ (heads/3.7:f328caf4ca, Mar 26 2018, 19:57:33) [Clang 9.0.0 (clang-900.0.39.2)] on darwin Type "help", "copyright", "credits" or "license" for more information. --> from enum import IntFlag --> class Flag(IntFlag): ... a = 1 ... b = 2 ... c = 4 ... d = 8 ... --> 'foo' in (Flag.a|Flag.b) True
Already being tracked at https://bugs.python.org/issue33217, which is what got me thinking about this whole issue. True is obviously wrong, but should the correct answer be False, or TypeError? -- ~Ethan~
I expect that for IntFlag `x in flags` is equivalent either to bool(x | flags.value) or to (x | flags.value) == x It should return some result only if x is an integer (or compatible with integers) and raise a TypeError otherwise. Don't add any special checks unless there are good reasons for this. Keep the code as simple as possible.
On 4/4/2018 11:32 AM, Ethan Furman wrote:
API question.
Background: ----------
When doing checks such as
--> 3 in [4, 5, 6] --> 'test' in {'test':True, 'live':False}
the result is True or False.
When doing checks such as
--> 3 in 'hello world' --> [4, 5, 6] in {'test':True, 'live':False}
the result is a TypeError.
The general rule seems to be that if it is impossible for the in-question object to be in the container object then a TypeError is raised, otherwise True or False is returned.
Question 1: ----------
(A) A standard Enum class is a container of Enum members. It cannot hold anything else. However, it has been returning False both in cases where the in-question object was an Enum of a different class (a Fruit in a Color, for example) and when the in-question object was not even an Enum ('apple' in Fruit, for example).
(B) The waters get even more muddied when Fruit has a str mixin, so `Fruit.APPLE == 'apple' is True` -- in that case, should `'orange' in Fruit` return True, False, or raise TypeError?
Question 2: ----------
(A) The new Flag type allows `in` tests on the members themselves; so, for example:
--> SomeFlag.ONE in SomeFlag.One|SomeFlag.TWO True
The question, of course, is what to do when a non-Flag member is tested for:
--> 'apple' in SomeFlag.ONE # False or TypeError?
--> 2 in SomeFlag.TWO # True or TypeError?
(B) And, of course, the same muddier question arises with IntFlag, where SomeFlag.TWO == 2 is True.
My thoughts: -----------
For question 1A (pure Enum): I'm thinking a TypeError should be raised when the in-question object is not an Enum member of any kind -- it simply is not possible for that object to ever be in an Enum, and is surely a bug.
For question 1B (mixed Enum): if 1A is TypeError, then 1B should also be TypeError at least for non-mixin in-question types (so checking for 1 in StrEnum would be a TypeError), but I'm torn between TypeError and True/False for cases where the in-question type matches the mixin type ('apple' in StrEnum).... On the one hand, even though an Enum member might be equal to some other type, that other type will not have the Enum attributes, etc, and a True answer would lead one to believe you could access `.name` and `.value`, etc., while a False answer would lead one to believe there was no match even though equality tests pass; on the other hand, how strong is the "container" aspect of a mixed Enum? How often is the test `'apple' in Fruit` meant to discover if you have a Fruit member vs whether you have something that could be a Fruit member? Also, how important is it to be consistent with IntFlag, which I definitely think should return True/False for int checks?
For question 2A (pure Flag): I'm comfortable sticking with a TypeError (assuming we switch to TypeError for 1A).
For question 2B (int Flag): I think a TypeError if the in-question object is not an int is appropriate (assuming TypeError for 1A and 2A), but if it is an int, True/False seems the better path. My reasoning being that Flag and IntFlag are more similar to sets than lists, and IntFlag is specifically meant to work with ints, and a test of `2 in some_int_flags` is more concerned with a flag being set than with .name or .value attributes that may or may not exist on the in-question object.
Any and all thoughts appreciated.
I think the "in" test should raise TypeError if tested against _anything_ that is not an Enum member. Why? I see a parallel between Enum and mappings. x = {'test':True, 'live':False} So it is True that 'test' in x and 'live' in x and False that True in x and False in x. It is False that 'foo' in x and 3 in x It is TypeError that [4,5,6] in x, or {'foo': 'bar'} in x. Note that it is False that (4,5,6) in x which is a little surprising given the above two, but not when you realize the differences between this and the above two. So with mappings, you can have any hashable type as a key: with Enum, you can only have Enum members as keys. So I have no idea why you would want to return False, rather than TypeError, other than (1) the distinction probably doesn't matter to most people (2) backward compatibility. I would find 2 in some_int_flags being True surprising.
On 04/04/2018 01:24 PM, Glenn Linderman wrote:
I think the "in" test should raise TypeError if tested against _anything_ that is not an Enum member.
Why? I see a parallel between Enum and mappings.
x = {'test':True, 'live':False}
So it is True that 'test' in x and 'live' in x and False that True in x and False in x.
It is False that 'foo' in x and 3 in x
It is TypeError that [4,5,6] in x, or {'foo': 'bar'} in x.
Note that it is False that (4,5,6) in x which is a little surprising given the above two, but not when you realize the differences between this and the above two.
So with mappings, you can have any hashable type as a key: with Enum, you can only have Enum members as keys.
Technically, an Enum has strings as keys (the member names), while the members themselves are the values; but list()ing an Enum returns the values (members) while a mapping would return the names (keys); Enums are definitely a bit strange!
So I have no idea why you would want to return False, rather than TypeError, other than (1) the distinction probably doesn't matter to most people (2) backward compatibility.
To me, `in` is an equality test, and equality tests should never, ever raise exceptions -- they should return True or False. Which is why I was surprised that `1 in "hello"` raised instead of returning False. However, I have learned that in certain situations, such as str or dict or set, if it is impossible to contain the item being searched for then raising an exception is appropriate. Hence my questions now on deciding what the behavior /should/ be, and whether practicality has a log to stand on in this case. ;) -- ~Ethan~
I don't think of "in" as an equality test -- it's special just like <, <=,
, >= are special, and those can certainly raise TypeError.
On Wed, Apr 4, 2018 at 2:07 PM, Ethan Furman
On 04/04/2018 01:24 PM, Glenn Linderman wrote:
I think the "in" test should raise TypeError if tested against _anything_
that is not an Enum member.
Why? I see a parallel between Enum and mappings.
x = {'test':True, 'live':False}
So it is True that 'test' in x and 'live' in x and False that True in x and False in x.
It is False that 'foo' in x and 3 in x
It is TypeError that [4,5,6] in x, or {'foo': 'bar'} in x.
Note that it is False that (4,5,6) in x which is a little surprising given the above two, but not when you realize the differences between this and the above two.
So with mappings, you can have any hashable type as a key: with Enum, you can only have Enum members as keys.
Technically, an Enum has strings as keys (the member names), while the members themselves are the values; but list()ing an Enum returns the values (members) while a mapping would return the names (keys); Enums are definitely a bit strange!
So I have no idea why you would want to return False, rather than
TypeError, other than (1) the distinction probably doesn't matter to most people (2) backward compatibility.
To me, `in` is an equality test, and equality tests should never, ever raise exceptions -- they should return True or False. Which is why I was surprised that `1 in "hello"` raised instead of returning False. However, I have learned that in certain situations, such as str or dict or set, if it is impossible to contain the item being searched for then raising an exception is appropriate.
Hence my questions now on deciding what the behavior /should/ be, and whether practicality has a log to stand on in this case. ;)
-- ~Ethan~ _______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido% 40python.org
-- --Guido van Rossum (python.org/~guido)
participants (6)
-
Barry Warsaw -
Brett Cannon -
Ethan Furman -
Glenn Linderman -
Guido van Rossum -
Serhiy Storchaka