PEP XXX - Competitor with PEP 435: Adding an enum type to the Python standard library
First, I offer my apologies to all who are still battle-weary from the last flurry of enum threads. However, while flufl.enum (PEP 435) is a good package for the use-case it handles, there are plenty of use-cases that it does not handle, or doesn't handle well, that it should not be the implementation in the stdlib. To quote one of my earlier emails:
This new PEP proposes an enum module that handles all those use cases, and makes it possible to handle others as well. If you recognize your idea but don't see your name in acknowledgements, please let me know. Code is available at https://bitbucket.org/stoneleaf/aenum -- ~Ethan~ ==============================================================================================
On 11 March 2013 16:18, Ethan Furman <ethan@stoneleaf.us> wrote:
One thing that is not discussed in the PEP (at least from my quick reading of it) is the issue of transitivity of equality (untested code, sorry, but I believe this is the intention of the PEP):
class E1(Enum): ... example = 1
class E2(Enum): ... example = 1
Personally, I find this behaviour unacceptable. The lack of a good way of dealing with this issue (I don't particularly like the "just use int()" or special value property approaches either) seems to be key to why people are failing to agree on an implementation... At the very least, there should be a section in the PEP addressing the discussion over this. The motivation section isn't particularly strong, either. There's nothing there that would persuade me to use enums. In general, the proposal seems geared mainly at people who *already* want to use enums, but for some reason aren't comfortable with using a 3rd party package. But maybe that just implies that I'm not part of the target audience for the feature, in which case fine (I don't have to use it, obviously...) Paul.
On 03/11/2013 09:55 AM, Paul Moore wrote:
There are four types you can inherit from: Enum, BitMask, Sequence, and String. Enum, BitMask, and String will not compare equal with integers, and Enum, BitMask and Sequence will not compare equal with strings; in fact, Enum and Bitmask, not being based on an existing data type, will not compare equal with anything that is not in their own enumeration group or its superclass. Sequences will compare equal with ints, because they are ints; they will also compare equal against other Sequence enumerations, as they are also ints. Same deal with Strings and strs. Those two are basically NamedValues in a fancy enum package.
Hopefully that's clearer now.
At the very least, there should be a section in the PEP addressing the discussion over this.
I'll get it added (probably next weekend, when I make any other necessary changes as well).
My main use case is an easy mapping from names to numbers, but that's not everyone's. -- ~Ethan~
On 11 March 2013 17:25, Ethan Furman <ethan@stoneleaf.us> wrote:
Yes, it is - thanks. I missed the significance of "Sequence" in the PEP (it's not a very obvious name for this functionality). Maybe there's a better name, or if not then at least making this aspect more prominent in the PEP would probably help. Paul
Paul Moore writes:
Yes, it is - thanks. I missed the significance of "Sequence" in the PEP
Overall the PEP's presentation suffers severely from class-instance confusion. In the *enumeration* Color(red, green, blue), Color.red is not an "enumeration", it is an *enumerator* (or element or member). Ditto for all the concrete class names. I'm not sure this really matters (I'm the kind of pedant who is enraged by the use of "it's" as a possessive), but it bugs me.<wink/> In the case of "Sequence", I would assume that the actual behavior is equivalent to range() with names. Based on past discussion of Ethan's use case, I suppose that that's not quite accurate. Rather, I expect that Sequence actually addresses the question of using an enumerator as an index to a Python sequence (list, etc.)
On 03/11/2013 05:20 PM, Stephen J. Turnbull wrote:
I would not be sad if you chose to go through and fix those things -- it seems I lack the necessary vocabulary. Although I usually get `its` correct. ;)
That's pretty much what I use it for, but it also solves Antoine's use case of, e.g., http status codes that otherwise act like normal ints. ~Ethan~
On Mon, Mar 11, 2013 at 10:25 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
First of all, thanks for working on this. It's healthy to have several approaches to solve the same problem. That said, I'm very much against the alternative you propose. The reason boils down to basic Pythonic principles. I imagine myself a few years from now reading Python 3.4+ code and seeing usage of these Enum, BitMask, Sequence (Integer?) and String classes, all slightly different in subtle ways, and that imaginary self will no doubt reach for the reference documentation on every occasion. That's because the 4 are similar yet different, and because they have no parallel in the C/C++ world (at least most of them don't). On the other hand, with flufl.enum, *because* it's so simple, it's very easy to grasp pretty immediately since it has few well defined uses cases that are similar in spirit to C's enum. Yes, in some cases I won't be able to use flufl.enum, and I'll fall back to the current "solution" of not having an enum package at all. But in the cases where I use it, I'll at least know that my code is becoming more readable. To summarize, my personal preference in priority order is: 1. get flufl.enum into stdlib, or a similarly simple proposal 2. don't get any enum package into stdlib at this point 3. get this alternative 4-class approach into stdlib Eli
On 03/12/2013 05:23 AM, Eli Bendersky wrote:
Would it be easier to accept if it was called "Enums and other Named Constants"? Currently, if I want to store a list of homogeneous values I can use: - list - tuple - bytes - bytearray - array.array Are these not "slightly different in subtle ways"? If simple is what we want, then do away with the "an enum is not an int" idea, toss out the "Color.red != OtherColor.red", and suddenly my four classes are down to two (int vs str), or flufl.enum suddenly handles many many more cases than it did before. But, quite frankly, I see value in having different enumerations being different types, and in having NamedConstants... perhaps some renaming would make things clearer? Currently: EnumType Enum(metaclass=EnumType) BitMask(metaclass=EnumType) Sequence(metaclass=EnumType) String(metaclass=EnumType) Could be instead: NamedConstantType Enum(metaclass=NamedConstantType) NamedInt(int, metaclass=NamedConstantType) NamedStr(str, metaclass=NamedConstantType) with available add-ons of BITMASK, INDEX, and ORDER. I don't know about you, but I like that a lot better. :) -- ~Ethan~
It is actually better, because it emphasizes that NamedInt is just that, not a kind of Enum. There's just one enum. Moreover, I'm not sure why strings need to be named (they name themselves just fine). And moreover+, Bitmask IMHO is completely unnecessary in Python. So if that leaves us with NamedInt / NamedFloat (in a similar vein to Nick's proposals) and Enum (with the semantics of PEP 435) I don't have major objections. Eli
On Mar 12, 2013, at 8:36, Eli Bendersky <eliben@gmail.com> wrote:
It is actually better, because it emphasizes that NamedInt is just that, not a kind of Enum. There's just one enum. Moreover, I'm not sure why strings need to be named (they name themselves just fine). And moreover+, Bitmask IMHO is completely unnecessary in Python.
It's necessary everywhere we interface with C APIs and binary formats that use them. Even the stdlib is full of candidates--the flags in os, stat, etc. are all bitmasks. I could see arguing that these shouldn't be considered the same kind of thing as an enum, or are too complex to handle in what should be a simple enum library. But I can't see how you can say we don't need them. The situation is identical to ordered named int--same status quo, same downsides to the status quo, etc.--except for the implementation complexity.
On Tue, Mar 12, 2013 at 9:59 AM, Andrew Barnert <abarnert@yahoo.com> wrote:
I think that viewing the Python programmer community at large, very few actually interact with C APIs that have bitmasked flags. Moreover, a NamedInt can fit the bill without needing a specific bitmask flag. If you have "names" for your flag constituents you can just join them with '|' as in C. This is similar to what's currently being done in modules like os and stat, but provides conveniently printable names for the magic numbers. The benefits of a specific bitmasking class in the stdlib are imho very marginal. Eli
From: Eli Bendersky <eliben@gmail.com> Sent: Tuesday, March 12, 2013 10:28 AM
I don't see why this even needs to be established. We have cases like, e.g., mmap.mmap all over the stdlib that take bitmasked flags. Are you arguing that these functions are too uncommon to belong in the stdlib or need to be redesigned? And, even if you don't touch any of those parts of the stdlib, there are many outside libraries that follow its example. From the "first steps" of the wx tutorial: window = wx.Frame(None, style=wx.MAXIMIZE_BOX | wx.RESIZE_BORDER | wx.SYSTEM_MENU | wx.CAPTION | wx.CLOSE_BOX) This isn't some design quirk of wx; this is how nearly every GUI framework except tkinter works. And image processing, audio processing, platform glue like win32api and PyObjC, and countless other application areas.
Moreover, a NamedInt can fit the bill without needing a specific bitmask flag.
If you have "names" for your flag constituents you can just join them with '|' as in C. This is similar to what's currently being done in modules like os and stat, but provides conveniently printable names for the magic numbers. The benefits of a specific bitmasking class in the stdlib are imho very marginal.
The benefits of a bitmasking enum class are exactly the same as the benefits of an ordered enum class. Compare: background, edge = color.RED, side.BOTTOM print(background, edge) print(background < edge) The benefits of NamedInt are that this doesn't print "1 4" and then "True". Now: style = wx.styles.MAXIMIZE_BOX | wx.styles.RESIZE_BORDER print(style) print(style & wx.keycodes.TAB) The benefits of a NamedInt are that this doesn't print "2098176" and then "True". If we don't have these benefits, there is no reason to add NamedInt in the first place, because it's nothing more than an alternate way to construct integer constants. Again, you could argue that making NamedInt work for the ordered case is trivial, making it work for the bitmask case is complicated, and therefore it's not worth doing even though it would be useful. But it clearly would be useful, for exactly the same reasons as in the ordered case.
On 03/12/2013 01:07 PM, Andrew Barnert wrote:
With a BitMask this could be: window = wx.Frame(None, style=wx.Style('MAXIMIZE_BOX|RESIZE_BORDER|SYSTEM_MENU|CAPTION|CLOSE_BOX'))
Hopefully you meant BitMask, as a NamedInt would result in True (unless either color or side were not NamedInts). While a BitMask is not directly an int, Barry showed how it turns into an int when passed to a C library.
How valuable is a name? And a doc string? Currently I'm working with OpenERP code, which uses constants in a lot of key places. I would love to know what 5 stands for, and 6. I knew 6 once, kind of, for about ten minutes while I worked with it, and now I've forgotten. -- ~Ethan~
Sigh.. I apologize for coming back in a bit late and still not really up to speed, but various real-world things ended up sidetracking me and over the past week or two and I haven't been able to spend as much time here as I'd like to.. I haven't yet had a chance to read the proposed PEP, but based on the discussions I'm seeing, I have to admit I'm a little disappointed that I thought we had already managed to come together with a pretty common list of properties that we all agreed on for enum-like things in Python, and I had started putting together a sample implementation based on those principles (which I never really heard anybody say anything substantially bad about, so I assumed it was sorta on the right track) and now it sounds like we're going off in significantly different directions again, and I'm not sure why. If this is in fact the case, it seems to me that crafting yet another PEP which doesn't mesh with half the community's expectations is not really useful and is probably premature, and we should probably go back to talking more about conceptually what we're actually trying to accomplish here first. FYI, I have actually been working on a newer version of my enum implementation which includes support for compound enums (i.e. "oring" enum values together), and I believe covers almost all of the properties that I've heard people express a desire for on this list, and avoids most of the things people seem to have major issues with, so I would be interested to know if people actually have problems with it. I'll see if I can get it cleaned up a bit and up on github in the next day or two so folks at least have an idea of how I've been looking at this stuff and can comment on what they think.. Regarding the issue of bitmask-enums, I do agree that they are common enough in various APIs that it is important that we be able to support them easily. *However,* I have yet to see how or why they are actually different than int-enums in any practical way. I don't see why we need to treat them as a different category at all and I see no value in doing so. They're all just* *int-enums. Problem solved. (I might have some thoughts on some of the rest of this stuff too, but I want to try to find a bit of time and read up on the new proposed-PEP and discussions before I respond further..) --Alex On Tue, Mar 12, 2013 at 1:40 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/12/2013 02:31 PM, Alex Stewart wrote:
Sadly, no. There seems to be two basic camps: those that think an enum should be valueless, and have nothing to do with an integer besides using it to select the appropriate enumerator (that just looks strange -- I hope you're right Stephen!); and those for whom the integer is an integral part of the enumeration, whether for sorting, comparing, selecting an index, or whatever. The critical aspect of using or not using an integer as the base type is: what happens when an enumerator from one class is compared to an enumerator from another class? If the base type is int and they both have the same value, they'll be equal -- so much for type safety; if the base type is object, they won't be equal, but then you lose your easy to use int aspect, your sorting, etc. Worse, if you have the base type be an int, but check for enumeration membership such that Color.red == 1 == Fruit.apple, but Color.red != Fruit.apple, you open a big can of worms because you just broke equality transitivity (or whatever it's called). We don't want that.
Looking forward to it.
What you gain with a dedicated BitMask is better reprs -- the same thing you gain with an enum, after all. Plus, with a standard enum you're using every number, but with a BitMask you are skipping most of them. --> class Color(Enum): ... black = 0 ... red = 1 ... green = 2 ... blue = 3 ... pink = 4 --> Color.red | Color.green # what do we see? Color.blue? Color.red|green? --> Color.blue | Color.pink # and what here? 7? Color.blue|Color.pink? -- ~Ethan~
On Tue, Mar 12, 2013 at 3:16 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
To be honest, I really don't think these two camps are as irreconcilable as lots of people seem to be treating them. I don't think either view is necessarily wrong, and I firmly believe that we can find a common ground that works for both ways of approaching this issue. More importantly, however, I believe we *must* try to find some way to coexist or anything we come up with will be fundamentally inadequate for a substantial portion of the community, and in that case, I don't think it belongs in the stdlib. If we can't compromise a bit, I think we're all doomed to failure. I do recognize that the issue of transitive equality is something we're going to need to work out. One thing I think folks need to keep in mind, though, is that that issue really has nothing to do with int-enums vs. valueless-enums at all. It will still be a point of contention even if we go with solely the "enums are ints" way of looking at things, so that really won't solve it. (The fundamental issue is: int-enums are (by definition) both ints, and named-objects. When not explicitly or implicitly being used as an int, should they still behave like ints first and named-objects second, or should they be named-objects first and ints second? Both ways have some problems, and we will ultimately need to choose which set of problems will be the least annoying in the long term, but that's a completely separate discussion than the int/valued/valueless question, IMHO.) --Alex
On Tue, Mar 12, 2013 at 4:42 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
There's a third camp - those that think the stdlib should just provide a basic explicit "labelled value" building block, and let others worry about composing them into more complex APIs and custom data types with defined semantics. Layered APIs are a good thing, and in this case, the obvious layering option is to separate out the "labelling" part which is broadly useful for debugging purposes from the "group of related values" part which is used as a tool to help ensure program correctness by triggering errors at the point of the undefined operation rather than allowing nonsensical results based on an underlying data type like int or str. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Tue, Mar 12, 2013 at 2:31 PM, Alex Stewart <foogod@gmail.com> wrote:
It just occurred to me that as I was skimming some of the previous discussion and I was also coming from the perspective of my own implementation, which has not yet been shared with the rest of the group, this response may have been overly terse and not really communicated what I meant very well. For clarification: I believe that what we're fundamentally talking about here is the question of "compounding" enum values into what are effectively multi-enum sets (they are not really the same thing anymore as a single enum value, because they do not have a one-to-one correspondence with any one enum value). With ints, this compounding operation is typically a "binary or" operation. This is what we call "bitmasks", but really they're just a particular way of compounding ints together. In my opinion, the concept of "compound enum" is a larger abstract concept that could, and probably should, apply to any type of enum, not just ints. In that context, bitmasks are really nothing special, they are just the compound form of int-enums: class Color (Enum): RED, GREEN, BLUE = __ * 3 class WxStyle (IntEnum): RESIZE_BORDER = 64 MAXIMIZE_BOX = 512
(This may become a bit clearer when you guys can all see the implementation I've been coding up) --Alex
On 03/12/2013 06:17 PM, Alex Stewart wrote:
So, what if each value of a 'flags' type enum is also able to be treated as a single-member set of itself? We already have one object that iterates into a sequence the same class: strings, so there's nothing wrong with that _conceptually_. The constructor could turn a sequence of the enum into a proper instance of it [if needed to get the int value] for passing into functions that require one, and a function that takes such a set of them could just not care the difference between Color.red|Color.blue and {Color.red,Color.blue}.
On 03/12/2013 06:03 PM, Random832 wrote:
In aenum.py you are able to iterate over a BitMask enum, both single ones and ORed together ones, even if it only has one value. The code looks like this: 8<---------------------------------------------------------------------- def __iter__(yo): enums = [] for enum in yo._enums: if int(enum) & int(yo): enums.append(enum) return iter(enums) 8<---------------------------------------------------------------------- -- ~Ethan~
From: Random832 <random832@fastmail.us>
That means that in order to make a window non-resizable, instead of this: window.style &= ~wx.RESIZE_BORDER you do this: window.style -= wx.RESIZE_BORDER Right? (IIRC, the actual code is probably something like "window.setStyle(window.getStyle() & ~wx.RESIZE_BORDER)", but I think we can ignore that; the issue is the same.) That solves the problem of how to represent ~wx.RESIZE_BORDER, etc. But it creates a new problem. Now non-bitmask integral Enum constants don't have integral-type operators, but set-type operators. So "FRIDAY - WEDNESDAY" returns "FRIDAY" (because it's set difference), and "WEDNESDAY < FRIDAY" is false (because it's not a proper subset)? So I think this means you _do_ need separate types for bitmasked and ordered int values—or, alternatively, for set-able and non-set-able Enums, which I think Alex Stewart was able to eliminate.
The constructor could turn a sequence of the enum into a proper instance of it
[if needed to get the int value] for passing into functions that require one,
Well, you need to be able to get the int value of a set, too. What's the point of being able to do mmap.PROT_READ | mmap.PROT_WRITE if I (or the mmap module) can't turn that into a 3? Also, what happens if you call the constructor on a sequence of more than one instance?
On 03/12/2013 09:58 PM, Andrew Barnert wrote:
Well, you need to be able to get the int value of a set, too. What's the point of being able to do mmap.PROT_READ | mmap.PROT_WRITE if I (or the mmap module) can't turn that into a 3?
No, you'd be able to turn _that_ into a 3. What I'm suggesting is being able to pass x = {t.PROT_READ, t.PROT_WRITE} into the mmap module, and then it can do int(t(x)) to get 3. Particularly since I don't really _like_ the a|b syntax for this, for some indefinable reason, and would like for passing in actual set literals to be the preferred way of doing it.
From: Alex Stewart <foogod@gmail.com> Sent: Tuesday, March 12, 2013 3:17 PM
Regarding the issue of bitmask-enums, I do agree that they are common enough in various APIs that it is important that we be able to support them easily. However, I have yet to see how or why they are actually different than int-enums in any practical way. I don't see why we need to treat them as a different category at all and I see no value in doing so. They're all just int-enums. Problem solved.
I agree with this. If there's an easy way to specify enum values, there's no reason that specifying 1, 2, 4, 8 has to be any different from specifying 0, 1, 2, 3. And, if there's a general-purpose way of "compounding enums" that lets me get the OR-ed value for passing to C/stdlib/etc. APIs that want that, I have no need for a special-purpose way of OR-ing bitmask enums. So, given all that, there is no value in treating them as a separate category. And it looks like maybe you've got a good answer that gives all that below, so…
For clarification: I believe that what we're fundamentally talking about here is the question of "compounding" enum values into what are effectively multi-enum sets (they are not really the same thing anymore as a single enum value, because they do not have a one-to-one correspondence with any one enum value). With ints, this compounding operation is typically a "binary or" operation. This is what we call "bitmasks", but really they're just a particular way of compounding ints together.
I suggested, a while back, that we could have some kind of EnumSet along with Enum, which sounds like what you're talking about here. So, if I did this: background = Color.RED | Color.BLUE What I get is not an Enum of type Color with a value RED | BLUE, but an EnumSet or type Color with a value {RED, BLUE}. Your CompoundEnum seems to be the same thing, except that it puts the "enum-ness" first instead of the "set-ness", which is probably an improvement. There are a few tricky bits here—most of which I didn't think through when I first suggested this, but it looks like you may have. You need a 0 value. After all, "background & Color.GREEN" has to return _something_. What is it called, what are its str and repr, etc.? You also need negated values, or there's no way to write "background &= ~Color.RED". So, "~Color.RED" has to be something. Presumably a CompoundEnum, with an int value of -2 (since RED is 1). But what are its str and repr? Does this mean internally CompoundEnum is a set of values and a set of negated values? If you want to make CompoundEnum work for string enums, what's the negation of a string value? Getting the list of interactions between Enum and CompoundEnum exactly right seems non-trivial. For example, "Color.RED in background" has to work even if background is just plain "Color.RED". (Otherwise, you'd have to write horrible stuff like "background == Color.RED or Color.RED in background".) You can't just say "both types support all numeric and set operations in the obvious way", because I don't think it's obvious that, e.g., __or__ treats either type as a set while __lt__ treats either type as a number. In fact, in some cases, I don't even know what the answer would be. Can __contains__ take a CompoundEnum on the left side? What's the __len__ of a CompoundEnum with negated values in it? And so on… Finally, what happens if someone has an int Enum whose values aren't unique bits, and tries to create a CompoundEnum out of it? If RED=1, BLUE=2, GREEN=3, is RED|BLUE an error? If so, that means that common cases like RDWR=3, ALL_FLAGS=0xFFFFFFFF, LOCAL_FLAGS=0x8FFC, etc. are errors too. On the other hand, if that's allowed, does that mean READ|WRITE gives me RDWR, or a CompoundEnum that compares equal to RDWR, …? For that matter, what about RDWR | ~READ? (And can I define ANTIRED=-2 so ~RED = ANTIRED?) If you've solved these problems, then yes, CompoundEnum completely eliminates the need for BitMask. And it sounds like, even if you haven't, you're at least well on your way to doing so. I'm looking forward to seeing it.
On Mar 12, 2013, at 19:04, Random832 <random832@fastmail.us> wrote:
Which means, as I said in another message, that Day.FRIDAY - Day.WEDNESDAY is not 2, but Day.FRIDAY. Which I'm pretty sure most people would find surprising. Unless you want bitmask and ordered int enums to be different types, I don't see a good way around this. A type that acts like a set and also like a number has conflicting intuitive meanings for most of its operators. The only reason you don't notice this for the obvious ones | and & is that those two set operations mean the same thing as the int operations (when thinking of an int as a set of bits); that's not true for any operators other than the bitwise ones.
On Wed, Mar 13, 2013, at 2:08, Andrew Barnert wrote:
That was part of my suggestion - I might not have been clear enough. Anyway - why would you subtract weekdays instead of subtracting the actual timestamps they're from and getting the number of days from the resulting timedelta? What makes you think a weekday enum wouldn't be a flag enum (so you can say an appointment is every monday, wednesday, and friday)?
On 11 March 2013 16:18, Ethan Furman <ethan@stoneleaf.us> wrote:
One thing that is not discussed in the PEP (at least from my quick reading of it) is the issue of transitivity of equality (untested code, sorry, but I believe this is the intention of the PEP):
class E1(Enum): ... example = 1
class E2(Enum): ... example = 1
Personally, I find this behaviour unacceptable. The lack of a good way of dealing with this issue (I don't particularly like the "just use int()" or special value property approaches either) seems to be key to why people are failing to agree on an implementation... At the very least, there should be a section in the PEP addressing the discussion over this. The motivation section isn't particularly strong, either. There's nothing there that would persuade me to use enums. In general, the proposal seems geared mainly at people who *already* want to use enums, but for some reason aren't comfortable with using a 3rd party package. But maybe that just implies that I'm not part of the target audience for the feature, in which case fine (I don't have to use it, obviously...) Paul.
On 03/11/2013 09:55 AM, Paul Moore wrote:
There are four types you can inherit from: Enum, BitMask, Sequence, and String. Enum, BitMask, and String will not compare equal with integers, and Enum, BitMask and Sequence will not compare equal with strings; in fact, Enum and Bitmask, not being based on an existing data type, will not compare equal with anything that is not in their own enumeration group or its superclass. Sequences will compare equal with ints, because they are ints; they will also compare equal against other Sequence enumerations, as they are also ints. Same deal with Strings and strs. Those two are basically NamedValues in a fancy enum package.
Hopefully that's clearer now.
At the very least, there should be a section in the PEP addressing the discussion over this.
I'll get it added (probably next weekend, when I make any other necessary changes as well).
My main use case is an easy mapping from names to numbers, but that's not everyone's. -- ~Ethan~
On 11 March 2013 17:25, Ethan Furman <ethan@stoneleaf.us> wrote:
Yes, it is - thanks. I missed the significance of "Sequence" in the PEP (it's not a very obvious name for this functionality). Maybe there's a better name, or if not then at least making this aspect more prominent in the PEP would probably help. Paul
Paul Moore writes:
Yes, it is - thanks. I missed the significance of "Sequence" in the PEP
Overall the PEP's presentation suffers severely from class-instance confusion. In the *enumeration* Color(red, green, blue), Color.red is not an "enumeration", it is an *enumerator* (or element or member). Ditto for all the concrete class names. I'm not sure this really matters (I'm the kind of pedant who is enraged by the use of "it's" as a possessive), but it bugs me.<wink/> In the case of "Sequence", I would assume that the actual behavior is equivalent to range() with names. Based on past discussion of Ethan's use case, I suppose that that's not quite accurate. Rather, I expect that Sequence actually addresses the question of using an enumerator as an index to a Python sequence (list, etc.)
On 03/11/2013 05:20 PM, Stephen J. Turnbull wrote:
I would not be sad if you chose to go through and fix those things -- it seems I lack the necessary vocabulary. Although I usually get `its` correct. ;)
That's pretty much what I use it for, but it also solves Antoine's use case of, e.g., http status codes that otherwise act like normal ints. ~Ethan~
On Mon, Mar 11, 2013 at 10:25 AM, Ethan Furman <ethan@stoneleaf.us> wrote:
First of all, thanks for working on this. It's healthy to have several approaches to solve the same problem. That said, I'm very much against the alternative you propose. The reason boils down to basic Pythonic principles. I imagine myself a few years from now reading Python 3.4+ code and seeing usage of these Enum, BitMask, Sequence (Integer?) and String classes, all slightly different in subtle ways, and that imaginary self will no doubt reach for the reference documentation on every occasion. That's because the 4 are similar yet different, and because they have no parallel in the C/C++ world (at least most of them don't). On the other hand, with flufl.enum, *because* it's so simple, it's very easy to grasp pretty immediately since it has few well defined uses cases that are similar in spirit to C's enum. Yes, in some cases I won't be able to use flufl.enum, and I'll fall back to the current "solution" of not having an enum package at all. But in the cases where I use it, I'll at least know that my code is becoming more readable. To summarize, my personal preference in priority order is: 1. get flufl.enum into stdlib, or a similarly simple proposal 2. don't get any enum package into stdlib at this point 3. get this alternative 4-class approach into stdlib Eli
On 03/12/2013 05:23 AM, Eli Bendersky wrote:
Would it be easier to accept if it was called "Enums and other Named Constants"? Currently, if I want to store a list of homogeneous values I can use: - list - tuple - bytes - bytearray - array.array Are these not "slightly different in subtle ways"? If simple is what we want, then do away with the "an enum is not an int" idea, toss out the "Color.red != OtherColor.red", and suddenly my four classes are down to two (int vs str), or flufl.enum suddenly handles many many more cases than it did before. But, quite frankly, I see value in having different enumerations being different types, and in having NamedConstants... perhaps some renaming would make things clearer? Currently: EnumType Enum(metaclass=EnumType) BitMask(metaclass=EnumType) Sequence(metaclass=EnumType) String(metaclass=EnumType) Could be instead: NamedConstantType Enum(metaclass=NamedConstantType) NamedInt(int, metaclass=NamedConstantType) NamedStr(str, metaclass=NamedConstantType) with available add-ons of BITMASK, INDEX, and ORDER. I don't know about you, but I like that a lot better. :) -- ~Ethan~
It is actually better, because it emphasizes that NamedInt is just that, not a kind of Enum. There's just one enum. Moreover, I'm not sure why strings need to be named (they name themselves just fine). And moreover+, Bitmask IMHO is completely unnecessary in Python. So if that leaves us with NamedInt / NamedFloat (in a similar vein to Nick's proposals) and Enum (with the semantics of PEP 435) I don't have major objections. Eli
On Mar 12, 2013, at 8:36, Eli Bendersky <eliben@gmail.com> wrote:
It is actually better, because it emphasizes that NamedInt is just that, not a kind of Enum. There's just one enum. Moreover, I'm not sure why strings need to be named (they name themselves just fine). And moreover+, Bitmask IMHO is completely unnecessary in Python.
It's necessary everywhere we interface with C APIs and binary formats that use them. Even the stdlib is full of candidates--the flags in os, stat, etc. are all bitmasks. I could see arguing that these shouldn't be considered the same kind of thing as an enum, or are too complex to handle in what should be a simple enum library. But I can't see how you can say we don't need them. The situation is identical to ordered named int--same status quo, same downsides to the status quo, etc.--except for the implementation complexity.
On Tue, Mar 12, 2013 at 9:59 AM, Andrew Barnert <abarnert@yahoo.com> wrote:
I think that viewing the Python programmer community at large, very few actually interact with C APIs that have bitmasked flags. Moreover, a NamedInt can fit the bill without needing a specific bitmask flag. If you have "names" for your flag constituents you can just join them with '|' as in C. This is similar to what's currently being done in modules like os and stat, but provides conveniently printable names for the magic numbers. The benefits of a specific bitmasking class in the stdlib are imho very marginal. Eli
From: Eli Bendersky <eliben@gmail.com> Sent: Tuesday, March 12, 2013 10:28 AM
I don't see why this even needs to be established. We have cases like, e.g., mmap.mmap all over the stdlib that take bitmasked flags. Are you arguing that these functions are too uncommon to belong in the stdlib or need to be redesigned? And, even if you don't touch any of those parts of the stdlib, there are many outside libraries that follow its example. From the "first steps" of the wx tutorial: window = wx.Frame(None, style=wx.MAXIMIZE_BOX | wx.RESIZE_BORDER | wx.SYSTEM_MENU | wx.CAPTION | wx.CLOSE_BOX) This isn't some design quirk of wx; this is how nearly every GUI framework except tkinter works. And image processing, audio processing, platform glue like win32api and PyObjC, and countless other application areas.
Moreover, a NamedInt can fit the bill without needing a specific bitmask flag.
If you have "names" for your flag constituents you can just join them with '|' as in C. This is similar to what's currently being done in modules like os and stat, but provides conveniently printable names for the magic numbers. The benefits of a specific bitmasking class in the stdlib are imho very marginal.
The benefits of a bitmasking enum class are exactly the same as the benefits of an ordered enum class. Compare: background, edge = color.RED, side.BOTTOM print(background, edge) print(background < edge) The benefits of NamedInt are that this doesn't print "1 4" and then "True". Now: style = wx.styles.MAXIMIZE_BOX | wx.styles.RESIZE_BORDER print(style) print(style & wx.keycodes.TAB) The benefits of a NamedInt are that this doesn't print "2098176" and then "True". If we don't have these benefits, there is no reason to add NamedInt in the first place, because it's nothing more than an alternate way to construct integer constants. Again, you could argue that making NamedInt work for the ordered case is trivial, making it work for the bitmask case is complicated, and therefore it's not worth doing even though it would be useful. But it clearly would be useful, for exactly the same reasons as in the ordered case.
On 03/12/2013 01:07 PM, Andrew Barnert wrote:
With a BitMask this could be: window = wx.Frame(None, style=wx.Style('MAXIMIZE_BOX|RESIZE_BORDER|SYSTEM_MENU|CAPTION|CLOSE_BOX'))
Hopefully you meant BitMask, as a NamedInt would result in True (unless either color or side were not NamedInts). While a BitMask is not directly an int, Barry showed how it turns into an int when passed to a C library.
How valuable is a name? And a doc string? Currently I'm working with OpenERP code, which uses constants in a lot of key places. I would love to know what 5 stands for, and 6. I knew 6 once, kind of, for about ten minutes while I worked with it, and now I've forgotten. -- ~Ethan~
Sigh.. I apologize for coming back in a bit late and still not really up to speed, but various real-world things ended up sidetracking me and over the past week or two and I haven't been able to spend as much time here as I'd like to.. I haven't yet had a chance to read the proposed PEP, but based on the discussions I'm seeing, I have to admit I'm a little disappointed that I thought we had already managed to come together with a pretty common list of properties that we all agreed on for enum-like things in Python, and I had started putting together a sample implementation based on those principles (which I never really heard anybody say anything substantially bad about, so I assumed it was sorta on the right track) and now it sounds like we're going off in significantly different directions again, and I'm not sure why. If this is in fact the case, it seems to me that crafting yet another PEP which doesn't mesh with half the community's expectations is not really useful and is probably premature, and we should probably go back to talking more about conceptually what we're actually trying to accomplish here first. FYI, I have actually been working on a newer version of my enum implementation which includes support for compound enums (i.e. "oring" enum values together), and I believe covers almost all of the properties that I've heard people express a desire for on this list, and avoids most of the things people seem to have major issues with, so I would be interested to know if people actually have problems with it. I'll see if I can get it cleaned up a bit and up on github in the next day or two so folks at least have an idea of how I've been looking at this stuff and can comment on what they think.. Regarding the issue of bitmask-enums, I do agree that they are common enough in various APIs that it is important that we be able to support them easily. *However,* I have yet to see how or why they are actually different than int-enums in any practical way. I don't see why we need to treat them as a different category at all and I see no value in doing so. They're all just* *int-enums. Problem solved. (I might have some thoughts on some of the rest of this stuff too, but I want to try to find a bit of time and read up on the new proposed-PEP and discussions before I respond further..) --Alex On Tue, Mar 12, 2013 at 1:40 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
On 03/12/2013 02:31 PM, Alex Stewart wrote:
Sadly, no. There seems to be two basic camps: those that think an enum should be valueless, and have nothing to do with an integer besides using it to select the appropriate enumerator (that just looks strange -- I hope you're right Stephen!); and those for whom the integer is an integral part of the enumeration, whether for sorting, comparing, selecting an index, or whatever. The critical aspect of using or not using an integer as the base type is: what happens when an enumerator from one class is compared to an enumerator from another class? If the base type is int and they both have the same value, they'll be equal -- so much for type safety; if the base type is object, they won't be equal, but then you lose your easy to use int aspect, your sorting, etc. Worse, if you have the base type be an int, but check for enumeration membership such that Color.red == 1 == Fruit.apple, but Color.red != Fruit.apple, you open a big can of worms because you just broke equality transitivity (or whatever it's called). We don't want that.
Looking forward to it.
What you gain with a dedicated BitMask is better reprs -- the same thing you gain with an enum, after all. Plus, with a standard enum you're using every number, but with a BitMask you are skipping most of them. --> class Color(Enum): ... black = 0 ... red = 1 ... green = 2 ... blue = 3 ... pink = 4 --> Color.red | Color.green # what do we see? Color.blue? Color.red|green? --> Color.blue | Color.pink # and what here? 7? Color.blue|Color.pink? -- ~Ethan~
On Tue, Mar 12, 2013 at 3:16 PM, Ethan Furman <ethan@stoneleaf.us> wrote:
To be honest, I really don't think these two camps are as irreconcilable as lots of people seem to be treating them. I don't think either view is necessarily wrong, and I firmly believe that we can find a common ground that works for both ways of approaching this issue. More importantly, however, I believe we *must* try to find some way to coexist or anything we come up with will be fundamentally inadequate for a substantial portion of the community, and in that case, I don't think it belongs in the stdlib. If we can't compromise a bit, I think we're all doomed to failure. I do recognize that the issue of transitive equality is something we're going to need to work out. One thing I think folks need to keep in mind, though, is that that issue really has nothing to do with int-enums vs. valueless-enums at all. It will still be a point of contention even if we go with solely the "enums are ints" way of looking at things, so that really won't solve it. (The fundamental issue is: int-enums are (by definition) both ints, and named-objects. When not explicitly or implicitly being used as an int, should they still behave like ints first and named-objects second, or should they be named-objects first and ints second? Both ways have some problems, and we will ultimately need to choose which set of problems will be the least annoying in the long term, but that's a completely separate discussion than the int/valued/valueless question, IMHO.) --Alex
On Tue, Mar 12, 2013 at 4:42 PM, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
There's a third camp - those that think the stdlib should just provide a basic explicit "labelled value" building block, and let others worry about composing them into more complex APIs and custom data types with defined semantics. Layered APIs are a good thing, and in this case, the obvious layering option is to separate out the "labelling" part which is broadly useful for debugging purposes from the "group of related values" part which is used as a tool to help ensure program correctness by triggering errors at the point of the undefined operation rather than allowing nonsensical results based on an underlying data type like int or str. Cheers, Nick. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia
On Tue, Mar 12, 2013 at 2:31 PM, Alex Stewart <foogod@gmail.com> wrote:
It just occurred to me that as I was skimming some of the previous discussion and I was also coming from the perspective of my own implementation, which has not yet been shared with the rest of the group, this response may have been overly terse and not really communicated what I meant very well. For clarification: I believe that what we're fundamentally talking about here is the question of "compounding" enum values into what are effectively multi-enum sets (they are not really the same thing anymore as a single enum value, because they do not have a one-to-one correspondence with any one enum value). With ints, this compounding operation is typically a "binary or" operation. This is what we call "bitmasks", but really they're just a particular way of compounding ints together. In my opinion, the concept of "compound enum" is a larger abstract concept that could, and probably should, apply to any type of enum, not just ints. In that context, bitmasks are really nothing special, they are just the compound form of int-enums: class Color (Enum): RED, GREEN, BLUE = __ * 3 class WxStyle (IntEnum): RESIZE_BORDER = 64 MAXIMIZE_BOX = 512
(This may become a bit clearer when you guys can all see the implementation I've been coding up) --Alex
On 03/12/2013 06:17 PM, Alex Stewart wrote:
So, what if each value of a 'flags' type enum is also able to be treated as a single-member set of itself? We already have one object that iterates into a sequence the same class: strings, so there's nothing wrong with that _conceptually_. The constructor could turn a sequence of the enum into a proper instance of it [if needed to get the int value] for passing into functions that require one, and a function that takes such a set of them could just not care the difference between Color.red|Color.blue and {Color.red,Color.blue}.
On 03/12/2013 06:03 PM, Random832 wrote:
In aenum.py you are able to iterate over a BitMask enum, both single ones and ORed together ones, even if it only has one value. The code looks like this: 8<---------------------------------------------------------------------- def __iter__(yo): enums = [] for enum in yo._enums: if int(enum) & int(yo): enums.append(enum) return iter(enums) 8<---------------------------------------------------------------------- -- ~Ethan~
From: Random832 <random832@fastmail.us>
That means that in order to make a window non-resizable, instead of this: window.style &= ~wx.RESIZE_BORDER you do this: window.style -= wx.RESIZE_BORDER Right? (IIRC, the actual code is probably something like "window.setStyle(window.getStyle() & ~wx.RESIZE_BORDER)", but I think we can ignore that; the issue is the same.) That solves the problem of how to represent ~wx.RESIZE_BORDER, etc. But it creates a new problem. Now non-bitmask integral Enum constants don't have integral-type operators, but set-type operators. So "FRIDAY - WEDNESDAY" returns "FRIDAY" (because it's set difference), and "WEDNESDAY < FRIDAY" is false (because it's not a proper subset)? So I think this means you _do_ need separate types for bitmasked and ordered int values—or, alternatively, for set-able and non-set-able Enums, which I think Alex Stewart was able to eliminate.
The constructor could turn a sequence of the enum into a proper instance of it
[if needed to get the int value] for passing into functions that require one,
Well, you need to be able to get the int value of a set, too. What's the point of being able to do mmap.PROT_READ | mmap.PROT_WRITE if I (or the mmap module) can't turn that into a 3? Also, what happens if you call the constructor on a sequence of more than one instance?
On 03/12/2013 09:58 PM, Andrew Barnert wrote:
Well, you need to be able to get the int value of a set, too. What's the point of being able to do mmap.PROT_READ | mmap.PROT_WRITE if I (or the mmap module) can't turn that into a 3?
No, you'd be able to turn _that_ into a 3. What I'm suggesting is being able to pass x = {t.PROT_READ, t.PROT_WRITE} into the mmap module, and then it can do int(t(x)) to get 3. Particularly since I don't really _like_ the a|b syntax for this, for some indefinable reason, and would like for passing in actual set literals to be the preferred way of doing it.
From: Alex Stewart <foogod@gmail.com> Sent: Tuesday, March 12, 2013 3:17 PM
Regarding the issue of bitmask-enums, I do agree that they are common enough in various APIs that it is important that we be able to support them easily. However, I have yet to see how or why they are actually different than int-enums in any practical way. I don't see why we need to treat them as a different category at all and I see no value in doing so. They're all just int-enums. Problem solved.
I agree with this. If there's an easy way to specify enum values, there's no reason that specifying 1, 2, 4, 8 has to be any different from specifying 0, 1, 2, 3. And, if there's a general-purpose way of "compounding enums" that lets me get the OR-ed value for passing to C/stdlib/etc. APIs that want that, I have no need for a special-purpose way of OR-ing bitmask enums. So, given all that, there is no value in treating them as a separate category. And it looks like maybe you've got a good answer that gives all that below, so…
For clarification: I believe that what we're fundamentally talking about here is the question of "compounding" enum values into what are effectively multi-enum sets (they are not really the same thing anymore as a single enum value, because they do not have a one-to-one correspondence with any one enum value). With ints, this compounding operation is typically a "binary or" operation. This is what we call "bitmasks", but really they're just a particular way of compounding ints together.
I suggested, a while back, that we could have some kind of EnumSet along with Enum, which sounds like what you're talking about here. So, if I did this: background = Color.RED | Color.BLUE What I get is not an Enum of type Color with a value RED | BLUE, but an EnumSet or type Color with a value {RED, BLUE}. Your CompoundEnum seems to be the same thing, except that it puts the "enum-ness" first instead of the "set-ness", which is probably an improvement. There are a few tricky bits here—most of which I didn't think through when I first suggested this, but it looks like you may have. You need a 0 value. After all, "background & Color.GREEN" has to return _something_. What is it called, what are its str and repr, etc.? You also need negated values, or there's no way to write "background &= ~Color.RED". So, "~Color.RED" has to be something. Presumably a CompoundEnum, with an int value of -2 (since RED is 1). But what are its str and repr? Does this mean internally CompoundEnum is a set of values and a set of negated values? If you want to make CompoundEnum work for string enums, what's the negation of a string value? Getting the list of interactions between Enum and CompoundEnum exactly right seems non-trivial. For example, "Color.RED in background" has to work even if background is just plain "Color.RED". (Otherwise, you'd have to write horrible stuff like "background == Color.RED or Color.RED in background".) You can't just say "both types support all numeric and set operations in the obvious way", because I don't think it's obvious that, e.g., __or__ treats either type as a set while __lt__ treats either type as a number. In fact, in some cases, I don't even know what the answer would be. Can __contains__ take a CompoundEnum on the left side? What's the __len__ of a CompoundEnum with negated values in it? And so on… Finally, what happens if someone has an int Enum whose values aren't unique bits, and tries to create a CompoundEnum out of it? If RED=1, BLUE=2, GREEN=3, is RED|BLUE an error? If so, that means that common cases like RDWR=3, ALL_FLAGS=0xFFFFFFFF, LOCAL_FLAGS=0x8FFC, etc. are errors too. On the other hand, if that's allowed, does that mean READ|WRITE gives me RDWR, or a CompoundEnum that compares equal to RDWR, …? For that matter, what about RDWR | ~READ? (And can I define ANTIRED=-2 so ~RED = ANTIRED?) If you've solved these problems, then yes, CompoundEnum completely eliminates the need for BitMask. And it sounds like, even if you haven't, you're at least well on your way to doing so. I'm looking forward to seeing it.
On Mar 12, 2013, at 19:04, Random832 <random832@fastmail.us> wrote:
Which means, as I said in another message, that Day.FRIDAY - Day.WEDNESDAY is not 2, but Day.FRIDAY. Which I'm pretty sure most people would find surprising. Unless you want bitmask and ordered int enums to be different types, I don't see a good way around this. A type that acts like a set and also like a number has conflicting intuitive meanings for most of its operators. The only reason you don't notice this for the obvious ones | and & is that those two set operations mean the same thing as the int operations (when thinking of an int as a set of bits); that's not true for any operators other than the bitwise ones.
On Wed, Mar 13, 2013, at 2:08, Andrew Barnert wrote:
That was part of my suggestion - I might not have been clear enough. Anyway - why would you subtract weekdays instead of subtracting the actual timestamps they're from and getting the number of days from the resulting timedelta? What makes you think a weekday enum wouldn't be a flag enum (so you can say an appointment is every monday, wednesday, and friday)?
participants (10)
-
Alex Stewart -
Andrew Barnert -
Eli Bendersky -
Ethan Furman -
Greg Ewing -
Nick Coghlan -
Paul Moore -
Random832 -
random832＠fastmail.us
-
Stephen J. Turnbull