I have a question about annotations for classes that derive from `Enum`. Instances of these classes are allowed to have custom instance variables that can be initialized by a custom `__init__` or `__new__` method. Refer to [this documentation](https://docs.python.org/3/library/enum.html#when-to-use-new-vs-init) for details. My question is, how would these instance variables be annotated in a type stub? How would a type checker differentiate between one of these instance variables and a class variable that represents a member of the enumeration (and therefore is typed as an instance of the enum class)? The metaclass apparently treats assignments within the class definition as enumeration values. Consider the following example: ```python class Color(Enum): # Members of the enumeration RED = ((1, 0, 0), "#FF0000") YELLOW = ((0, 1, 1), "#00FFFF") GREEN = ((0, 1, 0), "#00FF00") # Instance variables for instances of this enumeration components: Tuple[float, float, float] html_encoding: str # Custom init method def __init__(self, components: Tuple[float, float, float], html_encoding: str) -> None: self._value_ = html_value self.components = components self.html_encoding = html_encoding print(Color.RED.value) # "#FF0000" print(Color.RED.components) # (1, 0, 0) ``` How would I annotate this in a type stub? I would have thought the following: ```python class Color(Enum): RED = ... YELLOW = ... GREEN = ... components: Tuple[float, float, float] html_encoding: str ``` However, mypy interprets all five of the variables (including `components` and `html_encoding`) as members of the enumeration. ```python reveal_type(Color.RED) # Revealed type is 'Literal[test.Color.RED]?' reveal_type(Color.components) # Revealed type is 'Literal[test.Color.components]?' reveal_type(Color.html_encoding) # Revealed type is 'Literal[test.Color.html_encoding]?' ``` Furthermore, if I look at the way that enums are annotated in typeshed, they don't appear to use assignments to indicate enum elements. For example, `uuid.pyi` includes the following declaration. ```python class SafeUUID(Enum): safe: int unsafe: int unknown: None ``` So typeshed stubs appear to assume that variables with type annotations within a `Enum` class definition should be interpreted as members of the enumeration, not as instance variables within enum instances. I searched the typeshed and mypy issues and couldn't find any discussion of this topic. Any thoughts or suggestions? -- Eric Traut Contributor to Pyright & Pylance Microsoft Corp.
As a workaround, you can make you `Enum` class inherit from a dataclass: ```python from dataclasses import dataclass from enum import Enum @dataclass class MyClass: field1: int field2: str class MyEnum(MyClass, Enum): # Use enum field value to initialize MyClass instance MEMBER1 = 0, "zero" MEMBER2 = 42, "forty two" # enum member `value` is the raw initializer assert MyEnum.MEMBER1.value == (0, "zero") # enum member is an instance of `MyClass` assert MyEnum.MEMBER1.field1 == 0 assert MyEnum.MEMBER1.field2 == "zero" # Revealed type is still Literal of Enum member, but again, mypy knows that it's an instance of MyClass reveal_type(MyEnum.MEMBER1) # Revealed type is 'Literal[scratch.MyEnum.MEMBER1]?' # So dataclass members are correctly typed by mypy reveal_type(MyEnum.MEMBER1.field1) # Revealed type is 'builtins.int' # You can also use the raw enum value where an instance of MyClass is expected def func(arg: MyClass): ... func(MyEnum.MEMBER1) # No error here ``` However, there is one caveat: your dataclass (or NamedTuple, or custom class) cannot have fields named `name` or `value`, as they are reserved by `Enum`. In fact, this is just a workaround to override `__init__` method using the dataclass generated one. That's being said, it seems to me that mypy doesn't handle enum attribute declared in `__new__` (while Pycharm handle them correctly). But if you put field definition in `__init__` method, mypy will handle them: ```python from enum import Enum class MyEnum(Enum): def __init__(self, a: int, b: str): self.a = a self.b = b MEMBER = 0, "" reveal_type(MyEnum.MEMBER.b) # Revealed type is 'builtins.str' ``` And you can still add a `__new__` method where you only override the `_value_` if you need it. The advantage of using the `dataclass` workaround is again to reuse dataclass generated `init` instead of writing it. P.S. Maybe an issue should be opened in mypy repository to signal that `Enum.__new__` is not handled properly.
Thanks, but I'm not looking for a workaround. I'm looking for how this is intended to work so typeshed stubs and type checkers can handle this consistently. As I said above, I think the logical way to annotate an enum in a type stub would be: ```python class Color(Enum): RED = ... YELLOW = ... GREEN = ... components: Tuple[float, float, float] html_encoding: str ``` In other words, variables that use assignments are considered enumeration members, and variables that use a simple type annotation are instance variables. But that doesn't appear to match the assumptions in typeshed stubs, which use variables with type annotations to indicate enumeration members. I'm interested in hearing from maintainers of typeshed and the other major type checkers (mypy, pyre, pytype). -- Eric Traut Contributor to Pyright & Pylance Microsoft Corp.
El lun, 22 feb 2021 a las 13:24, Eric Traut (
Thanks, but I'm not looking for a workaround. I'm looking for how this is intended to work so typeshed stubs and type checkers can handle this consistently.
As I said above, I think the logical way to annotate an enum in a type stub would be: ```python class Color(Enum): RED = ... YELLOW = ... GREEN = ...
components: Tuple[float, float, float] html_encoding: str ```
In other words, variables that use assignments are considered enumeration members, and variables that use a simple type annotation are instance variables. But that doesn't appear to match the assumptions in typeshed stubs, which use variables with type annotations to indicate enumeration members.
I'm interested in hearing from maintainers of typeshed and the other major type checkers (mypy, pyre, pytype).
I feel like this is an area that wasn't specified in PEP 484 or other PEPs, so mypy ended up doing whatever made sense to implement, and typeshed went with whatever worked for mypy. It would be good to come up with a way to specify enums in stubs that can cover all use cases and work with all type checkers.
The syntax you propose unfortunately doesn't provide a way for the type checker to figure out the underlying type of the enum (i.e., the type of Color.RED.value; usually it's going to be int, but sometimes str). It's pretty common for users to need this.
-- Eric Traut Contributor to Pyright & Pylance Microsoft Corp. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: jelle.zijlstra@gmail.com
Jelle, that's a good point. One option is to special-case the `value` attribute and allow it to be annotated with the intended value type. That wouldn't allow different enumeration members to have different types though. I don't have any good suggestions for how to address this without introducing compatibility issues with existing stubs and type checkers. For now, I've updated pyright's logic to change its behavior based on whether it's analyzing a stub or non-stub file. If you're interested in the specifics, please refer to this issue in the pyright issue tracker: https://github.com/microsoft/pyright/issues/1521. -- Eric Traut Contributor to Pyright & Pylance Microsoft Corp.
I've been working on pytype's enum support. My goal is compatibility with
typeshed, which will end up looking like mypy's support: `components` and
`html_encoding` would end up as enum members, rather than fields on the
enum members.
This is acceptable to me, because my research showed most enums (in
Google's codebase, at least) are extremely simple and don't add extra
fields like this. That said, it would be ideal if we could support a wider
range of enums. I don't have a good suggestion for the syntax,
unfortunately.
-- Teddy
On Tue, Feb 23, 2021 at 8:09 AM Eric Traut
Jelle, that's a good point. One option is to special-case the `value` attribute and allow it to be annotated with the intended value type. That wouldn't allow different enumeration members to have different types though.
I don't have any good suggestions for how to address this without introducing compatibility issues with existing stubs and type checkers.
For now, I've updated pyright's logic to change its behavior based on whether it's analyzing a stub or non-stub file. If you're interested in the specifics, please refer to this issue in the pyright issue tracker: https://github.com/microsoft/pyright/issues/1521.
-- Eric Traut Contributor to Pyright & Pylance Microsoft Corp. _______________________________________________ Typing-sig mailing list -- typing-sig@python.org To unsubscribe send an email to typing-sig-leave@python.org https://mail.python.org/mailman3/lists/typing-sig.python.org/ Member address: tsudol@google.com
participants (4)
-
Eric Traut
-
Jelle Zijlstra
-
Joseph Perez
-
Teddy Sudol