[Python-ideas] PEP XXX - Competitor with PEP 435: Adding an enum type to the Python standard library
Ethan Furman
ethan at stoneleaf.us
Mon Mar 11 17:18:43 CET 2013
First, I offer my apologies to all who are still battle-weary from the last flurry of enum threads.
However, while flufl.enum (PEP 435) is a good package for the use-case it handles, there are plenty of use-cases that it
does not handle, or doesn't handle well, that it should not be the implementation in the stdlib. To quote one of my
earlier emails:
> I'm beginning to see why enums as a class has not yet been added to Python.
> We don't want to complicate the language with too many choices, yet there is
> no One Obvious Enum to fit the wide variety of use-cases:
>
> - named int enums (http status codes)
> - named str enums (tkinter options)
> - named bitmask enums (file-type options)
> - named valueless enums (any random set of names)
> - named valueless-yet-orderable enums (any not-so-random set of names ;)
This new PEP proposes an enum module that handles all those use cases, and makes it possible to handle others as well.
If you recognize your idea but don't see your name in acknowledgements, please let me know.
Code is available at https://bitbucket.org/stoneleaf/aenum
--
~Ethan~
==============================================================================================
-------------- next part --------------
PEP: xxx
Title: Adding an Enum type to the Python standard library
Version: $Revision$
Last-Modified: $Date$
Author: Ethan Furman <ethan at stoneleaf.us>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2013-03-08
Python-Version: 3.4
Post-History: 2013-03-08
Abstract
========
This PEP proposes adding an enumeration type to the Python standard library.
Portions are derived or excerpted from PEP-0435.
An enumeration is a set of symbolic names bound to unique, constant integer
values. Depending on the circumstances, the exact integer can be integral
to the enums use (the name is simply an easy way to refer to the number) or
the integer value may simply serve as a way to select which enumeration to
use (e.g. to store or retrieve from a database).
Motivation
==========
*[Based partly on the Motivation stated in PEP 354 and 435]*
The properties of an enumeration are useful for defining an immutable, related
set of constant values. Classic examples are days of the week (Sunday through
Saturday) and school assessment grades ('A' through 'D', and 'F'). Other
examples include error status values and states within a defined process.
It is possible to simply define a sequence of values of some other basic type,
such as ``int`` or ``str``, to represent discrete arbitrary values. However,
an enumeration ensures that such values are distinct from any others including,
importantly, values within other enumerations, and that operations without
meaning ("Wednesday times two") are not defined for these values. It also
provides a convenient printable representation of enum values without requiring
tedious repetition while defining them (i.e. no ``GREEN = 'green'``).
Rationale
=========
Discussions to add enumerations to Python regularly occur, yet one has yet
to be added; I suspect this is due to the wide range of use cases::
- named int enums (http status codes)
- named str enums (tkinter options)
- named bitmask enums (file-type options)
- named valueless enums (any random set of names)
- named valueless-yet-orderable enums (any not so random set of names)
Clearly, to have one Enum fulfill all those roles would make for complicated,
hard to maintain code.
Alternatives
============
flufl.enum has been proposed by Guido::
pros: established, well-tested, handles its use-case very well (value-less,
unordered)
cons: only handles one of the use cases well, with some support for a second
(named int enums)
Tim Delaney, Alex Stewart, and myself have come up with largely similar,
alternate enum implementations that better support more of the above use cases..
Proposal
========
This PEP proposes that the enumeration implementation Aenum be accepted as
Python's stdlib enum.
Aenum has one base Enum, three derived Enums --BitMask, Sequence, and String--
and two options (ORDER and INDEX) to allow users to easily create their own
extended types in the few cases where these do not meet their needs.
Module and type name
====================
I propose to add a module named ``enum`` to the standard library. The main
type exposed by this module is ``Enum``, with subtypes ``Sequence``, ``String``,
and ``BitMask``, the options ``INDEX`` and ``ORDER``, and one helper class,
``enum``.
``Enum` - a valueless, unordered type. It's related integer value is merely to
allow for database storage and selection from the enumerated class. An
``Enum`` will not compare equal with its integer value, but can compare equal
to other enums of which it is a subclass.
``Sequence`` - a named ``int``. The enumerated name is merely to identify the value,
and all normal integer operations are supported, but the resulting
value will not be an enumeration. A ``Sequence`` will compare equal to its
integer value; in fact, it /is/ its integer value, and any mathematical
operations will return an ``int``, not an ``enum``.
``String`` - a named ``str``. The enumerated name may be a shortcut for a longer
string, but by default the enumerated name is the string it represents. Like
``Enum``s, ``String``s do not compare equal to its integer value; but similar to
``Sequence``s, ``String``s act just like ``str``, and any operations will return
a ``str``, not an ``enum``.
``BitMask`` - an ``Enum`` that supports the bitwise operations, and whose integer
values are matched accordingly (0, 1, 2, 4, 8, 16, etc.).
``INDEX`` - adding this option adds the ``__index__`` magic method to the enum class;
the associated integer is returned.
``ORDER`` - adding this option adds the ``__lt__``, ``__le__``, ``__ge__``, and ``__gt__``
magic methods.
``enum`` - supports auto-numbering enumerated names, as well as assigning other
attributes/values to the ``enum`` instance.
Proposed semantics for the new enumeration type
===============================================
Creating an Enum
----------------
Enumerations are primarily created using the class syntax, which makes them
easy to read and write. Every enumeration value must have a unique integer
value and the only restriction on their names is that they must be valid Python
identifiers. To define an enumeration, derive from the ``Enum``, ``BitMask``,
``Sequence``, or ``String`` classes and add attributes either with assignment
to their integer values, or with assignment of the ``enum()`` helper that
supports auto-numbering::
>>> from enum import Enum
>>> class Colors(Enum):
... black = 0
... red = 1
... green = 2
... blue = 3
or::
>>> from enum import Enum, enum
>>> class Colors(Enum):
... black = enum() # auto-numbered at 0
... red = enum() # at 1
... green = enum() # at 2
... blue = enum() # at 3
>>> from enum import BitMask, enum
>>> class FieldOptions(BitMask):
... none = enum() # auto-numbered at 0
... binary = enum() # at 1
... auto_inc = enum() # at 2
... unique = enum() # at 4
... nullable = enum() # at 8
>>> from enum import Sequence, enum
>>> class HttpStatus(Sequence):
... ok = enum(doc="request fulfilled", integer=200)
... created = enum(doc="POST success", integer=201)
... redirect = enum(doc="permanent redirect", integer=301)
... forbidden = enum(doc="authorization will not help", integer=403)
... not_implemented = enum(doc="service is not implemented", integer=501)
>>> from enum import String, enum
>>> class TkLocation(String):
... N = enum(value='north') # auto-numbered at 0
... S = enum(value='south') # 1
... E = enum(value='east') # 2
... W = enum(value='west') # 3
Equal and not-equal work for all enumerations::
>>> Colors.red == Colors.red
True
>>> FieldOptions.binary != HttpStatus.created
True
>>> TkLocation.N == 'north' # String enums are instances of str!
True
>>> HttpStatus.redirect == 301 # Sequence enums are instances of int!
True
Less-than, less-than-or-equal, greater-than, and greater-than-or-equal only
work for ordered enums:
>>> HttpStatus.ok < HttpStatus.forbidden
True
>>> TkLocation.N > TkLocation.E # String enums are str, so ordering
False # is str-based
Unordered enums raise an exception:
>>> Colors.red < Colors.blue
Traceback (most recent call last):
...
TypeError: unorderable types: Color() < Color()
But you can add order if you need it:
>>> from enum import Enum, ORDER
>>> class Grades(Enum, EnumOptions=ORDER):
... A = enum(doc="Excellent", integer=5)
... B = enum(doc="Above Average", integer=4)
... C = enum(doc="Average", integer=3)
... D = enum(doc="Below Average", integer=2)
... F = enum(doc="Insufficient", integer=1)
>>> Grades.A > Grades.B
True
>>> Grades.D < Grades.C
True
>>> Grades.F > Grades.C
False
Typically, enumerations will not compare equal to either their integer value nor their string name::
>>> Colors.red == 1
False
>>> FieldOptions.nullable == 8
False
>>> TkLocation.E == 3
False
>>> Colors.blue == 'blue'
False
>>> FieldOptions.unique == 'unique'
False
>>> HttpStatus.ok == 'ok'
False
>>> TkLocation.N == 'N'
False
Unless the enum is Sequence or a default String:
>>> HttpStatus.ok == 200
True
>>> class TkManager(String):
... grid = enum() # auto-numbered at 0, auto-valued at 'grid'
... pack = enum() # auto-numbered at 1, auto-valued at 'pack'
>>> TkManager.pack == 'pack'
True
Enumeration values have nice, human readable string representations::
>>> print(Colors.red)
Colors.red
...while their repr has more information::
>>> print(repr(Colors.red))
Colors("red", integer=1)
The enumeration instances are available through the class::
>>> for color in Colors:
... print(color)
black
red
green
blue
Enums also have a property that contains just their item name::
>>> print(Colors.black.__name__)
black
>>> print(Colors.red.__name__)
red
>>> print(Colors.green.__name__)
green
>>> print(Colors.blue.__name__)
blue
The str and repr of the enumeration class also provides useful information::
>>> print(Colors)
'Colors(black=0, red=1, green=2, blue=3)'
>>> print(repr(Colors))
'Colors(black=0, red=1, green=2, blue=3)'
You can extend previously defined Enums by subclassing::
>>> class MoreColors(Colors):
... cyan = 4
... magenta = 5
... yellow = 6
When extended in this way, the base enumeration's values are identical to the
same named values in the derived class::
>>> Colors.red == MoreColors.red
True
>>> Colors.blue == MoreColors.blue
True
However, if you define an enumeration that is not subclassing, with similar
item names and/or integer values, they will not be identical::
>>> class OtherColors(Enum):
... red = 1
... blue = 2
... yellow = 3
>>> Colors.red == OtherColors.red
False
>>> Colors.blue != OtherColors.blue
True
>>> MoreColors.yellow == OtherColors.yellow
False
These enumerations are not equal, nor do they hash equally::
>>> Colors.red == OtherColors.red
False
>>> len(set((Colors.red, OtherColors.red)))
2
When you need the integer equivalent values, you can convert enumerations
explicitly using the ``int()`` built-in. This is quite convenient for
storing enums in a database, as well as for interoperability with C extensions
that expect integers::
>>> int(colors.black)
0
>>> int(Colors.red)
1
>>> int(Colors.green)
2
>>> int(Colors.blue)
3
You can also convert back to the enumeration value by calling the Enum
subclass, passing in the integer value for the item you want::
>>> Colors(0)
Colors("black", integer=0)
>>> Colors(3)
Colors("blue", integer=3)
>>> Colors(1) == Colors.red
True
The Enum subclass also accepts the string name of the enumeration value::
>>> Colors('green')
Colors("green", integer=2)
>>> Colors('blue') == Colors.blue
True
You get exceptions though, if you try to use invalid arguments::
>>> Colors('magenta')
Traceback (most recent call last):
...
enum.InvalidEnum: magenta is not a valid Color
>>> Colors(99)
Traceback (most recent call last):
...
enum.InvalidEnum: 99 is not a valid Color
ValueError: 99
The integer equivalent values serve another purpose. You may not define two
enumeration values with the same integer value::
>>> class Bad(Enum):
... cartman = 1
... stan = 2
... kyle = 3
... kenny = 3 # Oops!
... butters = 4
Traceback (most recent call last):
...
TypeError: Multiple enum values: 3
You also may not duplicate values in derived enumerations::
>>> class BadColors(Colors):
... yellow = 4
... chartreuse = 2 # Oops!
Traceback (most recent call last):
...
TypeError: Multiple enum values: 2
The Enum class support iteration. Enumeration values are returned in the
sorted order of their integer equivalent values::
>>> [v.__name__ for v in MoreColors]
['black', 'red', 'green', 'blue', 'cyan', 'magenta', 'yellow']
>>> [int(v) for v in MoreColors]
[0, 1, 2, 3, 4, 5, 6]
Enumeration values are hashable, so they can be used in dictionaries and sets::
>>> apples = {}
>>> apples[Colors.red] = 'red delicious'
>>> apples[Colors.green] = 'granny smith'
>>> for color in sorted(apples, key=int):
... print(color.name, '->', apples[color])
red -> red delicious
green -> granny smith
Pickling (not yet implemented)
--------
Enumerations created with the class syntax can also be pickled and unpickled::
>>> from enum.tests.fruit import Fruit
>>> from pickle import dumps, loads
>>> Fruit.tomato is loads(dumps(Fruit.tomato))
True
Convenience API
---------------
You can also create enumerations using the class method ``create()``,
which takes either a string of space-separated names or an iterable object of
``(name, value)`` pairs.
The first argument to ``create()`` is the name of the enumeration. The second
argument is a *source* which can be either a string of space-separated names,
or an iterable of ``(name, value)`` pairs. In the most basic usage, *source*
is a sequence of strings which name the enumeration items. In this case, the
values are automatically assigned starting from 0::
>>> from enum import Enum
>>> Enum.create('Animals', 'ant bee cat dog'))
<Animals {ant: 1, bee: 2, cat: 3, dog: 4}>
The items in source can also be 2-tuples, where the first item is the
enumeration value name and the second is the integer value to assign to the
value. If 2-tuples are used, all items must be 2-tuples::
>>> from enum import Sequence
>>> Sequence.create('Row', (('name', 0), ('mobile', 1), ('email', 2), ('url', 3))
Row(name=0, mobile=1, email=2, url=3
Proposed variations
===================
Some variations were proposed during the discussions in the mailing list.
Here's some of the more popular ones.
Not having to specify values for enums
--------------------------------------
Michael Foord proposed (and Tim Delaney provided a proof-of-concept
implementation) to use metaclass magic that makes this possible::
class Color(Enum):
red, green, blue
The values get actually assigned only when first looked up.
Pros: cleaner syntax that requires less typing for a very common task (just
listing enumeration names without caring about the values).
Cons: involves much magic in the implementation, which makes even the
definition of such enums baffling when first seen. If a name is duplicated the
magic won't see it (as it already exists) so it won't get the next integer,
which can lead to hard to find bugs.
Using special names or forms to auto-assign enum values
-------------------------------------------------------
A different approach to avoid specifying enum values is to use a special name
or form to auto assign them. For example::
class Color(Enum):
red = None # auto-assigned to 0
green = None # auto-assigned to 1
blue = None # auto-assigned to 2
More flexibly::
class Color(Enum):
red = 7
green = None # auto-assigned to 8
blue = 19
purple = None # auto-assigned to 20
Some variations on this theme:
#. A special name ``auto`` imported from the enum package.
#. Georg Brandl proposed ellipsis (``...``) instead of ``None`` to achieve the
same effect.
Pros: no need to manually enter values. Makes it easier to change the enum and
extend it, especially for large enumerations.
Cons: actually longer to type in many simple cases. The argument of explicit
vs. implicit applies here as well.
The variation that won
----------------------
While only saving on typing if you use cut and paste, the winning idea was to
use a helper function, ``enum()``, that would allow the integer to not be
specified. It also allows for other arbitrary attributes to be set on the
enumerated values (most importantly, a doc string).
Use-cases in the standard library
=================================
The Python standard library has many places where the usage of enums would be
beneficial to replace other idioms currently used to represent them. Such
usages can be divided to two categories: user-code facing constants, and
internal constants.
User-code facing constants like ``os.SEEK_*``, ``socket`` module constants,
decimal rounding modes, HTML error codes, etc., would benefit by being
converted to enums. Because they are now either ``int``s or ``str``s the
``Sequence`` or ``String`` type enums would have to be used, as they are
interchangeable with ``int``s and ``str``s and thus would not break backward-
compatibility.
Internal constants are not seen by user code but are employed internally by
stdlib modules. It appears that nothing should stand in the way of
implementing such constants with the standard Enum. Some examples uncovered
by a very partial skim through the stdlib: ``binhex``, ``imaplib``,
``http/client``, ``urllib/robotparser``, ``idlelib``, ``concurrent.futures``,
``turtledemo``.
In addition, looking at the code of the Twisted library, there are many use
cases for replacing internal state constants with enums. The same can be said
about a lot of networking code (especially implementation of protocols) and
can be seen in test protocols written with the Tulip library as well.
Differences from PEP 354 and 435
================================
Unlike PEP 354, enumeration values are not defined as a sequence of strings,
but as attributes of a class. This design was chosen because it was felt that
class syntax is more readable.
Unlike PEP 354, enumeration values require an integer value. This difference
recognizes that enumerations often represent real-world values, or must
interoperate with external real-world systems. For example, to store an
enumeration in a database, it is better to convert it to an integer on the way
in and back to an enumeration on the way out. Providing an integer value also
provides an explicit ordering.
Unlike PEP 354, this implementation does use a metaclass to define the
enumeration's syntax, and allows for extended base-enumerations so that the
common values in derived classes are comparable.
Like PEP 435 enumerations within a class are singletons, but unlike PEP 435
subclassed values, while comparing equal, are not the same object.
Acknowledgments
===============
This PEP describes the ``aenum`` module by Ethan Furman. ``aenum`` is based
on examples and ideas from Michael Foord, Tim Delaney, Alex Stewart, Yuri
Selivanov and ``flufl.enum`` by Barry Warsaw. Ben Finney is the author of the
earlier enumeration PEP 354 [1]_, and Barry Warsaw and Eli Bendersky are the authors
of the competing PEP 435 [2]_.
References
==========
.. [1] http://www.python.org/dev/peps/pep-0354/
.. [2] http://www.python.org/dev/peps/pep-0435/
Copyright
=========
This document has been placed in the public domain.
Todo
====
* Mark PEP 354 "superseded by" this one, if accepted
* New package name within stdlib - enum? (top-level)
* Verify (and add support if need be) that other base-types will
function as enums; e.g. ``class Constants(float, metaclass=EnumType)``
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
More information about the Python-ideas
mailing list