[Python-Dev] PEP 435 - requesting pronouncement
Eli Bendersky
eliben at gmail.com
Sun May 5 00:04:49 CEST 2013
Hello pydev,
PEP 435 is ready for final review. A lot of the feedback from the last few
weeks of discussions has been incorporated. Naturally, not everything could
go in because some minor (mostly preference-based) issues did not reach a
consensus. We do feel, however, that the end result is better than in the
beginning and that Python can finally have a useful enumeration type in the
standard library.
I'm attaching the latest version of the PEP for convenience. If you've read
previous versions, the easiest way to get acquainted with the recent
changes is to go through the revision log at http://hg.python.org/peps
A reference implementation for PEP 435 is available at
https://bitbucket.org/stoneleaf/ref435
Kind regards and happy weekend.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130504/d33560a1/attachment.html>
-------------- next part --------------
PEP: 435
Title: Adding an Enum type to the Python standard library
Version: $Revision$
Last-Modified: $Date$
Author: Barry Warsaw <barry at python.org>,
Eli Bendersky <eliben at gmail.com>,
Ethan Furman <ethan at stoneleaf.us>
Status: Draft
Type: Standards Track
Content-Type: text/x-rst
Created: 2013-02-23
Python-Version: 3.4
Post-History: 2013-02-23, 2013-05-02
Abstract
========
This PEP proposes adding an enumeration type to the Python standard library.
An enumeration is a set of symbolic names bound to unique, constant values.
Within an enumeration, the values can be compared by identity, and the
enumeration itself can be iterated over.
Decision
========
TODO: update decision here once pronouncement is made. [1]_
Status of discussions
=====================
The idea of adding an enum type to Python is not new - PEP 354 [2]_ is a
previous attempt that was rejected in 2005. Recently a new set of discussions
was initiated [3]_ on the ``python-ideas`` mailing list. Many new ideas were
proposed in several threads; after a lengthy discussion Guido proposed adding
``flufl.enum`` to the standard library [4]_. During the PyCon 2013 language
summit the issue was discussed further. It became clear that many developers
want to see an enum that subclasses ``int``, which can allow us to replace
many integer constants in the standard library by enums with friendly string
representations, without ceding backwards compatibility. An additional
discussion among several interested core developers led to the proposal of
having ``IntEnum`` as a special case of ``Enum``.
The key dividing issue between ``Enum`` and ``IntEnum`` is whether comparing
to integers is semantically meaningful. For most uses of enumerations, it's
a **feature** to reject comparison to integers; enums that compare to integers
lead, through transitivity, to comparisons between enums of unrelated types,
which isn't desirable in most cases. For some uses, however, greater
interoperatiliby with integers is desired. For instance, this is the case for
replacing existing standard library constants (such as ``socket.AF_INET``)
with enumerations.
Further discussion in late April 2013 led to the conclusion that enumeration
members should belong to the type of their enum: ``type(Color.red) == Color``.
Guido has pronounced a decision on this issue [5]_, as well as related issues
of not allowing to subclass enums [6]_, unless they define no enumeration
members [7]_.
Motivation
==========
*[Based partly on the Motivation stated in PEP 354]*
The properties of an enumeration are useful for defining an immutable, related
set of constant values that have a defined sequence but no inherent semantic
meaning. Classic examples are days of the week (Sunday through Saturday) and
school assessment grades ('A' through 'D', and 'F'). Other examples include
error status values and states within a defined process.
It is possible to simply define a sequence of values of some other basic type,
such as ``int`` or ``str``, to represent discrete arbitrary values. However,
an enumeration ensures that such values are distinct from any others including,
importantly, values within other enumerations, and that operations without
meaning ("Wednesday times two") are not defined for these values. It also
provides a convenient printable representation of enum values without requiring
tedious repetition while defining them (i.e. no ``GREEN = 'green'``).
Module and type name
====================
We propose to add a module named ``enum`` to the standard library. The main
type exposed by this module is ``Enum``. Hence, to import the ``Enum`` type
user code will run::
>>> from enum import Enum
Proposed semantics for the new enumeration type
===============================================
Creating an Enum
----------------
Enumerations are created using the class syntax, which makes them easy to read
and write. An alternative creation method is described in `Functional API`_.
To define an enumeration, subclass ``Enum`` as follows::
>>> from enum import Enum
>>> class Color(Enum):
... red = 1
... green = 2
... blue = 3
**A note on nomenclature**: we call ``Color`` an *enumeration* (or *enum*)
and ``Color.red``, ``Color.green`` are *enumeration members* (or
*enum members*). Enumeration members also have *values* (the value of
``Color.red`` is ``1``, etc.)
Enumeration members have human readable string representations::
>>> print(Color.red)
Color.red
...while their ``repr`` has more information::
>>> print(repr(Color.red))
<Color.red: 1>
The *type* of an enumeration member is the enumeration it belongs to::
>>> type(Color.red)
<Enum 'Color'>
>>> isinstance(Color.green, Color)
True
>>>
Enums also have a property that contains just their item name::
>>> print(Color.red.name)
red
Enumerations support iteration, in definition order::
>>> class Shake(Enum):
... vanilla = 7
... chocolate = 4
... cookies = 9
... mint = 3
...
>>> for shake in Shake:
... print(shake)
...
Shake.vanilla
Shake.chocolate
Shake.cookies
Shake.mint
Enumeration members are hashable, so they can be used in dictionaries and sets::
>>> apples = {}
>>> apples[Color.red] = 'red delicious'
>>> apples[Color.green] = 'granny smith'
>>> apples
{<Color.red: 1>: 'red delicious', <Color.green: 2>: 'granny smith'}
Programmatic access to enumeration members
------------------------------------------
Sometimes it's useful to access members in enumerations programmatically (i.e.
situations where ``Color.red`` won't do because the exact color is not known
at program-writing time). ``Enum`` allows such access::
>>> Color(1)
<Color.red: 1>
>>> Color(3)
<Colro.blue: 3>
If you want to access enum members by *name*, use item access::
>>> Color['red']
<Color.red: 1>
>>> Color['green']
<Color.green: 2>
Duplicating enum members and values
-----------------------------------
Having two enum members with the same name is invalid::
>>> class Shape(Enum):
... square = 2
... square = 3
...
Traceback (most recent call last):
...
TypeError: Attempted to reuse key: square
However, two enum members are allowed to have the same value. Given two members
A and B with the same value (and A defined first), B is an alias to A. By-value
lookup of the value of A and B will return A.
>>> class Shape(Enum):
... square = 2
... diamond = 1
... circle = 3
... alias_for_square = 2
...
>>> Shape.square
<Shape.square: 2>
>>> Shape.alias_for_square
<Shape.square: 2>
>>> Shape(2)
<Shape.square: 2>
Iterating over the members of an enum does not provide the aliases::
>>> list(Shape)
[<Shape.square: 2>, <Shape.diamond: 1>, <Shape.circle: 3>]
If access to aliases is required for some reason, use the special attribute
``__aliases__``::
>>> Shape.__aliases__
['alias_for_square']
Comparisons
-----------
Enumeration members are compared by identity::
>>> Color.red is Color.red
True
>>> Color.red is Color.blue
False
>>> Color.red is not Color.blue
True
Ordered comparisons between enumeration values are *not* supported. Enums are
not integers (but see `IntEnum`_ below)::
>>> Color.red < Color.blue
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unorderable types: Color() < Color()
Equality comparisons are defined though::
>>> Color.blue == Color.red
False
>>> Color.blue == Color.blue
True
Comparisons against non-enumeration values will always compare not equal
(again, ``IntEnum`` was explicitly designed to behave differently, see
below)::
>>> Color.blue == 2
False
Allowed members and attributes of enumerations
----------------------------------------------
The examples above use integers for enumeration values. Using integers is
short and handy (and provided by default by the `Functional API`_), but not
strictly enforced. In the vast majority of use-cases, one doesn't care what
the actual value of an enumeration is. But if the value *is* important,
enumerations can have arbitrary values.
Enumerations are Python classes, and can have methods and special methods as
usual. If we have this enumeration::
class Mood(Enum):
funky = 1
happy = 3
def describe(self):
# self is the member here
return self.name, self.value
def __str__(self):
return 'my custom str! {0}'.format(self.value)
@classmethod
def favorite_mood(cls):
# cls here is the enumeration
return cls.happy
Then::
>>> Mood.favorite_mood()
<Mood.happy: 3>
>>> Mood.happy.describe()
('happy', 3)
>>> str(Mood.funky)
'my custom str! 1'
The rules for what is allowed are as follows: all attributes defined within an
enumeration will become members of this enumeration, with the exception of
*__dunder__* names and descriptors; methods are descriptors too.
Restricted subclassing of enumerations
--------------------------------------
Subclassing an enumeration is allowed only if the enumeration does not define
any members. So this is forbidden::
>>> class MoreColor(Color):
... pink = 17
...
TypeError: Cannot extend enumerations
But this is allowed::
>>> class Foo(Enum):
... def some_behavior(self):
... pass
...
>>> class Bar(Foo):
... happy = 1
... sad = 2
...
The rationale for this decision was given by Guido in [6]_. Allowing to
subclass enums that define members would lead to a violation of some
important invariants of types and instances. On the other hand, it
makes sense to allow sharing some common behavior between a group of
enumerations, and subclassing empty enumerations is also used to implement
``IntEnum``.
IntEnum
-------
A variation of ``Enum`` is proposed which is also a subclass of ``int``.
Members of an ``IntEnum`` can be compared to integers; by extension,
integer enumerations of different types can also be compared to each other::
>>> from enum import IntEnum
>>> class Shape(IntEnum):
... circle = 1
... square = 2
...
>>> class Request(IntEnum):
... post = 1
... get = 2
...
>>> Shape == 1
False
>>> Shape.circle == 1
True
>>> Shape.circle == Request.post
True
However they still can't be compared to ``Enum``::
>>> class Shape(IntEnum):
... circle = 1
... square = 2
...
>>> class Color(Enum):
... red = 1
... green = 2
...
>>> Shape.circle == Color.red
False
``IntEnum`` values behave like integers in other ways you'd expect::
>>> int(Shape.circle)
1
>>> ['a', 'b', 'c'][Shape.circle]
'b'
>>> [i for i in range(Shape.square)]
[0, 1]
For the vast majority of code, ``Enum`` is strongly recommended,
since ``IntEnum`` breaks some semantic promises of an enumeration (by
being comparable to integers, and thus by transitivity to other
unrelated enumerations). It should be used only in special cases where
there's no other choice; for example, when integer constants are
replaced with enumerations and backwards compatibility is required
with code that still expects integers.
Other derived enumerations
--------------------------
``IntEnum`` will be part of the ``enum`` module. However, it would be very
simple to implement independently::
class IntEnum(int, Enum):
pass
This demonstrates how similar derived enumerations can be defined, for example
a ``StrEnum`` that mixes in ``str`` instead of ``int``.
Some rules:
1. When subclassing Enum, mixing types must appear before Enum itself in the
sequence of bases.
2. While Enum can have members of any type, once you mix in an additional
type, all the members must have values of that type, e.g. ``int`` above.
This restriction does not apply to behavior-only mixins.
Functional API
--------------
The ``Enum`` class is callable, providing the following functional API::
>>> Animal = Enum('Animal', 'ant bee cat dog')
>>> Animal
<Enum 'Animal'>
>>> Animal.ant
<Animal.ant: 1>
>>> Animal.ant.value
1
>>> list(Animal)
[<Animal.ant: 1>, <Animal.bee: 2>, <Animal.cat: 3>, <Animal.dog: 4>]
The semantics of this API resemble ``namedtuple``. The first argument of
the call to ``Enum`` is the name of the enumeration. The second argument is
a source of enumeration member names. It can be a whitespace-separated string
of names, a sequence of names or a sequence of 2-tuples with key/value pairs.
The last option enables assigning arbitrary values to enumerations; the others
auto-assign increasing integers starting with 1. A new class derived from
``Enum`` is returned. In other words, the above assignment to ``Animal`` is
equivalent to::
>>> class Animals(Enum):
... ant = 1
... bee = 2
... cat = 3
... dog = 4
Pickling
--------
Enumerations be pickled and unpickled::
>>> from enum.tests.fruit import Fruit
>>> from pickle import dumps, loads
>>> Fruit.tomato is loads(dumps(Fruit.tomato))
True
The usual restrictions for pickling apply: picklable enums must be defined in
the top level of a module, to be importable from that module when unpickling
occurs.
Proposed variations
===================
Some variations were proposed during the discussions in the mailing list.
Here's some of the more popular ones.
flufl.enum
----------
``flufl.enum`` was the reference implementation upon which this PEP was
originally based. Eventually, it was decided against the inclusion of
``flufl.enum`` because its design separated enumeration members from
enumerations, so the former are not instances of the latter. Its design
also explicitly permits subclassing enumerations for extending them with
more members (due to the member/enum separation, the type invariants are not
violated in ``flufl.enum`` with such a scheme).
Not having to specify values for enums
--------------------------------------
Michael Foord proposed (and Tim Delaney provided a proof-of-concept
implementation) to use metaclass magic that makes this possible::
class Color(Enum):
red, green, blue
The values get actually assigned only when first looked up.
Pros: cleaner syntax that requires less typing for a very common task (just
listing enumeration names without caring about the values).
Cons: involves much magic in the implementation, which makes even the
definition of such enums baffling when first seen. Besides, explicit is
better than implicit.
Using special names or forms to auto-assign enum values
-------------------------------------------------------
A different approach to avoid specifying enum values is to use a special name
or form to auto assign them. For example::
class Color(Enum):
red = None # auto-assigned to 0
green = None # auto-assigned to 1
blue = None # auto-assigned to 2
More flexibly::
class Color(Enum):
red = 7
green = None # auto-assigned to 8
blue = 19
purple = None # auto-assigned to 20
Some variations on this theme:
#. A special name ``auto`` imported from the enum package.
#. Georg Brandl proposed ellipsis (``...``) instead of ``None`` to achieve the
same effect.
Pros: no need to manually enter values. Makes it easier to change the enum and
extend it, especially for large enumerations.
Cons: actually longer to type in many simple cases. The argument of explicit
vs. implicit applies here as well.
Use-cases in the standard library
=================================
The Python standard library has many places where the usage of enums would be
beneficial to replace other idioms currently used to represent them. Such
usages can be divided to two categories: user-code facing constants, and
internal constants.
User-code facing constants like ``os.SEEK_*``, ``socket`` module constants,
decimal rounding modes and HTML error codes could require backwards
compatibility since user code may expect integers. ``IntEnum`` as described
above provides the required semantics; being a subclass of ``int``, it does not
affect user code that expects integers, while on the other hand allowing
printable representations for enumeration values::
>>> import socket
>>> family = socket.AF_INET
>>> family == 2
True
>>> print(family)
SocketFamily.AF_INET
Internal constants are not seen by user code but are employed internally by
stdlib modules. These can be implemented with ``Enum``. Some examples
uncovered by a very partial skim through the stdlib: ``binhex``, ``imaplib``,
``http/client``, ``urllib/robotparser``, ``idlelib``, ``concurrent.futures``,
``turtledemo``.
In addition, looking at the code of the Twisted library, there are many use
cases for replacing internal state constants with enums. The same can be said
about a lot of networking code (especially implementation of protocols) and
can be seen in test protocols written with the Tulip library as well.
Acknowledgments
===============
This PEP was initially proposing including the ``flufl.enum`` package [8]_
by Barry Warsaw into the stdlib, and is inspired in large parts by it.
Ben Finney is the author of the earlier enumeration PEP 354.
References
==========
.. [1] Placeholder for pronouncement
.. [2] http://www.python.org/dev/peps/pep-0354/
.. [3] http://mail.python.org/pipermail/python-ideas/2013-January/019003.html
.. [4] http://mail.python.org/pipermail/python-ideas/2013-February/019373.html
.. [5] http://mail.python.org/pipermail/python-dev/2013-April/125687.html
.. [6] http://mail.python.org/pipermail/python-dev/2013-April/125716.html
.. [7] http://mail.python.org/pipermail/python-dev/2013-May/125859.html
.. [8] http://pythonhosted.org/flufl.enum/
Copyright
=========
This document has been placed in the public domain.
Todo
====
* Mark PEP 354 "superseded by" this one, if accepted
* The last revision where flufl.enum was the approach is cb3c18a080a3
..
Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
coding: utf-8
End:
More information about the Python-Dev
mailing list