PEP 663: Improving and Standardizing Enum str(), repr(), and format() behaviors
PEP 663 is presented below for your viewing pleasure. Comments, questions, and concerns are welcome. -------------------------------------------------------------------------------- PEP: 663 Title: Improving and Standardizing Enum str(), repr(), and format() behaviors Version: $Revision$ Last-Modified: $Date$ Author: Ethan Furman <ethan@stoneleaf.us> Discussions-To: python-dev@python.org Status: Draft Type: Informational Content-Type: text/x-rst Created: 23-Feb-2013 Python-Version: 3.11 Post-History: 20-Jul-2021 10-Sep-2021 Resolution: Abstract ======== Now that we have a few years experience with Enum usage it is time to update the ``repr()``, ``str()``, and ``format()`` of the various enumerations by their intended purpose. Motivation ========== The addition of ``StrEnum`` with its requirement to have its ``str()`` be its ``value`` is inconsistent with other provided Enum's ``str``. Having the ``str()`` of ``IntEnum`` and ``IntFlag`` not be the value causes bugs and extra work when replacing existing constants. Having the ``str()`` and ``format()`` of an enum member be different can be confusing. The iteration of ``Flag`` members, which directly affects their ``repr()``, is inelegant at best, and buggy at worst. Rationale ========= Enums are becoming more common in the standard library; being able to recognize enum members by their ``repr()``, and having that ``repr()`` be easy to parse, is useful and can save time and effort in understanding and debugging code. However, the enums with mixed-in data types (``IntEnum``, ``IntFlag``, and the new ``StrEnum``) need to be more backwards compatible with the constants they are replacing -- specifically, ``str(replacement_enum_member) == str(original_constant)`` should be true (and the same for ``format()``). IntEnum, IntFlag, and StrEnum should be as close to a drop-in replacement of existing integer and string constants as is possible. Towards that goal, the str() output of each should be its inherent value; e.g. if ``Color`` is an ``IntEnum``:: >>> Color.RED <Color.RED: 1> >>> str(Color.RED) '1' >>> format(Color.RED) '1' Note that format() already produces the correct output in 3.10, only str() needs updating. As much as possible, the ``str()`, ``repr()``, and ``format()`` of enum members should be standardized across the stardard library. The repr() of Flag currently includes aliases, which it should not; fixing that will, of course, already change its ``repr()`` in certain cases. Specification ============= There a three broad categories of enum usage: - standard: Enum or Flag a new enum class is created, and the members are used as ``class.member_name`` - drop-in replacement: IntEnum, IntFlag, StrEnum a new enum class is created which also subclasses ``int`` or ``str`` and uses ``int.__str__`` or ``str.__str__``; the ``repr`` can be changed by using the ``global_enum`` decorator - user-mixed enums and flags the user creates their own integer-, float-, str-, whatever-enums instead of using enum.IntEnum, etc. Some sample enums:: # module: tools.py class Hue(Enum): # or IntEnum LIGHT = -1 NORMAL = 0 DARK = +1 class Color(Flag): # or IntFlag RED = 1 GREEN = 2 BLUE = 4 class Grey(int, Enum): # or (int, Flag) BLACK = 0 WHITE = 1 Using the above enumerations, the following table shows the old and new behavior, while the last shows the final result: +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | type | enum repr() | enum str() | enum format() | flag repr() | flag str() | flag format() | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | standard | 3.10 | | | | <Color.RED|GREEN: 3> | Color.RED|GREEN | Color.RED|GREEN | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | | | <Color(3): RED|GREEN> | Color.RED|Color.GREEN | Color.RED|Color.GREEN | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | 3.10 | | | 1 | <Grey.WHITE: 1> | | 1 | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | | Grey.WHITE | <Grey(1): WHITE> | | Grey.WHITE | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | 3.10 | | Hue.LIGHT | | <Color.RED|GREEN: 3> | Color.RED|GREEN | | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | -1 | | <Color(3): RED|GREEN> | 3 | | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | global | 3.10 | <Hue.LIGHT: -1> | Hue.LIGHT | Hue.LIGHT | <Color.RED|GREEN: 3> | Color.RED|GREEN | Color.RED|GREEN | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.LIGHT | LIGHT | LIGHT | tools.RED|tools.GREEN | RED|GREEN | RED|GREEN | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | 3.10 | <Grey.WHITE: 1 | Grey.WHITE | Grey.WHITE | <Grey.WHITE: 1> | Grey.WHITE | 1 | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.WHITE | WHITE | WHITE | tools.WHITE | WHITE | WHITE | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | 3.10 | <Hue.LIGHT: -1> | Hue.LIGHT | | <Color.RED|GREEN: 3> | Color.RED|GREEN | | | +----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.LIGHT | -1 | | tools.RED|tools.GREEN | 3 | | +-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ Which will result in: +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | type | enum repr() | enum str() | enum format() | flag repr() | flag str() | flag format() | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | standard | <Hue.LIGHT: -1> | Hue.LIGHT | Hue.LIGHT | <Color(3): RED|GREEN> | Color.RED|Color.GREEN | Color.RED|Color.GREEN | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | <Grey.WHITE: 1> | Grey.WHITE | Grey.WHITE | <Grey(1): WHITE> | Grey.WHITE | Grey.WHITE | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | <Hue.LIGHT: -1> | -1 | -1 | <Color(3): RED|GREEN> | 3 | 3 | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | global | tools.LIGHT | LIGHT | LIGHT | tools.RED|tools.GREEN | RED|GREEN | RED|GREEN | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | tools.WHITE | WHITE | WHITE | tools.WHITE | WHITE | WHITE | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | tools.LIGHT | -1 | -1 | tools.RED|tools.GREEN | 3 | 3 | +-------------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ As can be seen, ``repr()`` is primarily affected by whether the members are global, while ``str()`` is affected by being global or by being a drop-in replacement, with the drop-in replacement status having a higher priority. Also, the basic ``repr()`` and ``str()`` have changed for flags as the old style was very clunky. The ``repr()`` for Enum vs Flag are different, primarily because the Enum ``repr()`` does not work well for flags. I like being able to tell whether an enum member is a Flag or an Enum based on the ``repr()`` alone, but am open to arguments for changing Enum's ``repr()`` to match Flag's. Backwards Compatibility ======================= Backwards compatibility of stringified objects is not guarenteed across major Python versions, and there will be backwards compatibility breaks where software uses the ``repr()``, ``str()``, and ``format()`` output of enums in tests, documentation, data structures, and/or code generation. Normal usage of enum members will not change: ``re.ASCII`` can still be used as ``re.ASCII`` and will still compare equal to ``256``. If the previous output needs to be maintained, for example to ensure compatibily between different Python versions, software projects will need to create their own enum base class with the appropriate methods overridden. Note that by changing the ``str()`` of the drop-in category, we will actually prevent future breakage when ``IntEnum``, et al, are used to replace existing constants. Copyright ========= This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
On 11 Sep 2021, at 06:57, Ethan Furman <ethan@stoneleaf.us> wrote:
If the previous output needs to be maintained, for example to ensure compatibily between different Python versions, software projects will need to create their own enum base class with the appropriate methods overridden.
Perhaps you could provide that backwards-compatible base class, either in the library as a deprecated class or at least in the PEP for people to copy if they need it?
On 9/11/2021 6:53 AM, Ethan Furman wrote:
Abstract ========
Now that we have a few years experience with Enum usage it is time to update the ``repr()``, ``str()``, and ``format()`` of the various enumerations by their intended purpose.
You *are* allowed to put some (brief) details in the abstract. No need to avoid spoilers ;) As it stands, "it is time" on its own is a really bad reason to change anything. So you're preemptively making it sound like the PEP has no solid backing. I haven't read the rest yet. The abstract is supposed to make me want to read it, and this one does not, so I stopped. (Might come back later when I'm not trying to catch up on my weekend's email though.) Cheers, Steve
On 9/13/21 9:03 AM, Steve Dower wrote:
You *are* allowed to put some (brief) details in the abstract. No need to avoid spoilers ;)
As it stands, "it is time" on its own is a really bad reason to change anything. So you're preemptively making it sound like the PEP has no solid backing.
Abstract ======== Update the ``repr()``, ``str()``, and ``format()`` of the various Enum types for consistency and to better match their intended purpose. Better? -- ~Ethan~
On 9/13/2021 8:12 PM, Ethan Furman wrote:
On 9/13/21 9:03 AM, Steve Dower wrote:
You *are* allowed to put some (brief) details in the abstract. No need to avoid spoilers ;)
As it stands, "it is time" on its own is a really bad reason to change anything. So you're preemptively making it sound like the PEP has no solid backing.
Abstract ========
Update the ``repr()``, ``str()``, and ``format()`` of the various Enum types for consistency and to better match their intended purpose.
Better?
You don't have a one sentence summary of what the changes entail? "Originally these were based on the idea that ..., but now will work better for ... by making the results more consistent with ..." (where each "..." is filled with specific things). It doesn't have to save me reading the whole thing, but if I'm digging through documents going "why am I seeing <Red:1> instead of just '1'", it should confirm that this is the right document to read. (Alternatively, think about writing the tweet here that you want people to include when they share your PEP, assuming that 99% of people won't actually click through to the document. What can you tell them so that they at least know what is coming?) Cheers, Steve
El vie, 10 sept 2021 a las 22:56, Ethan Furman (<ethan@stoneleaf.us>) escribió:
<snip>
Specification =============
There a three broad categories of enum usage:
- standard: Enum or Flag a new enum class is created, and the members are used as ``class.member_name``
- drop-in replacement: IntEnum, IntFlag, StrEnum a new enum class is created which also subclasses ``int`` or ``str`` and uses ``int.__str__`` or ``str.__str__``; the ``repr`` can be changed by using the ``global_enum`` decorator
- user-mixed enums and flags the user creates their own integer-, float-, str-, whatever-enums instead of using enum.IntEnum, etc.
Some sample enums::
# module: tools.py
class Hue(Enum): # or IntEnum LIGHT = -1 NORMAL = 0 DARK = +1
class Color(Flag): # or IntFlag RED = 1 GREEN = 2 BLUE = 4
class Grey(int, Enum): # or (int, Flag) BLACK = 0 WHITE = 1
Using the above enumerations, the following table shows the old and new behavior, while the last shows the final result:
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | type | enum repr() | enum str() | enum format() | flag repr() | flag str() | flag format() |
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | standard | 3.10 | | | | <Color.RED|GREEN: 3> | Color.RED|GREEN | Color.RED|GREEN | |
+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | | | <Color(3): RED|GREEN> | Color.RED|Color.GREEN | Color.RED|Color.GREEN |
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | 3.10 | | | 1 | <Grey.WHITE: 1> | | 1 | |
+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | | Grey.WHITE | <Grey(1): WHITE> | | Grey.WHITE |
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | 3.10 | | Hue.LIGHT | | <Color.RED|GREEN: 3> | Color.RED|GREEN | | |
+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | | -1 | | <Color(3): RED|GREEN> | 3 | |
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | global | 3.10 | <Hue.LIGHT: -1> | Hue.LIGHT | Hue.LIGHT | <Color.RED|GREEN: 3> | Color.RED|GREEN | Color.RED|GREEN | |
+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.LIGHT | LIGHT | LIGHT | tools.RED|tools.GREEN | RED|GREEN | RED|GREEN |
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | user mixed | 3.10 | <Grey.WHITE: 1 | Grey.WHITE | Grey.WHITE | <Grey.WHITE: 1> | Grey.WHITE | 1 | |
+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.WHITE | WHITE | WHITE | tools.WHITE | WHITE | WHITE |
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | int drop-in | 3.10 | <Hue.LIGHT: -1> | Hue.LIGHT | | <Color.RED|GREEN: 3> | Color.RED|GREEN | | |
+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+ | | new | tools.LIGHT | -1 | | tools.RED|tools.GREEN | 3 | |
+-------------+----------+-----------------+------------+-----------------------+-----------------------+------------------------+-----------------------+
This table doesn't render properly in https://www.python.org/dev/peps/pep-0663/#specification. - There are some extra blank lines - Many cells are blank - There are two "user mixed" rows and it's not clear how those relate to the mention of "three broad categories" above.
On 9/13/21 9:55 AM, Jelle Zijlstra wrote:
This table doesn't render properly in https://www.python.org/dev/peps/pep-0663/#specification. - There are some extra blank lines - Many cells are blank - There are two "user mixed" rows and it's not clear how those relate to the mention of "three broad categories" above.
The blank cells are the "no change" cells; the blank lines were there to separate the types (removed). I have reworked those tables slightly -- hopefully they make more sense now. -- ~Ethan~
participants (4)
-
Ethan Furman -
Irit Katriel -
Jelle Zijlstra -
Steve Dower