Mailman 3 August 2020 - Python-checkins

bpo-40979: refactored typing.rst; (mostly) same content, new sub-sections and ordering (#21574)
by Luciano Ramalho Aug. 2, 2020

Aug. 2, 2020

https://github.com/python/cpython/commit/ab72fdeb82c3ab045b480cd4bb4f928c12… commit: ab72fdeb82c3ab045b480cd4bb4f928c12653ecb branch: master author: Luciano Ramalho <luciano(a)ramalho.org> committer: GitHub <noreply(a)github.com> date: 2020-08-02T15:32:36-07:00 summary: bpo-40979: refactored typing.rst; (mostly) same content, new sub-sections and ordering (#21574) Also added PEP 585 deprecation notes. files: A Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst M Doc/library/typing.rst diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index e5143c5f8d9e4..44b537f1669e1 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -1,3 +1,4 @@ +======================================== :mod:`typing` --- Support for type hints ======================================== @@ -34,7 +35,7 @@ In the function ``greeting``, the argument ``name`` is expected to be of type arguments. Type aliases ------------- +============ A type alias is defined by assigning the type to the alias. In this example, ``Vector`` and ``List[float]`` will be treated as interchangeable synonyms:: @@ -72,7 +73,7 @@ Note that ``None`` as a type hint is a special case and is replaced by .. _distinct: NewType -------- +======= Use the :func:`NewType` helper function to create distinct types:: @@ -149,7 +150,7 @@ See :pep:`484` for more details. .. versionadded:: 3.5.2 Callable --------- +======== Frameworks expecting callback functions of specific signatures might be type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``. @@ -172,7 +173,7 @@ for the list of arguments in the type hint: ``Callable[..., ReturnType]``. .. _generics: Generics --------- +======== Since type information about objects kept in containers cannot be statically inferred in a generic way, abstract base classes have been extended to support @@ -199,7 +200,7 @@ called :class:`TypeVar`. User-defined generic types --------------------------- +========================== A user-defined class can be defined as a generic class. @@ -317,7 +318,7 @@ comparable for equality. The :data:`Any` type --------------------- +==================== A special kind of type is :data:`Any`. A static type checker will treat every type as being compatible with :data:`Any` and :data:`Any` as being @@ -395,7 +396,7 @@ manner. Use :data:`Any` to indicate that a value is dynamically typed. Nominal vs structural subtyping -------------------------------- +=============================== Initially :pep:`484` defined Python static type system as using *nominal subtyping*. This means that a class ``A`` is allowed where @@ -434,106 +435,152 @@ Moreover, by subclassing a special class :class:`Protocol`, a user can define new custom protocols to fully enjoy structural subtyping (see examples below). +Module contents +=============== -Classes, functions, and decorators ----------------------------------- +The module defines the following classes, functions and decorators. -The module defines the following classes, functions and decorators: +.. note:: -.. class:: TypeVar + This module defines several types that are subclasses of pre-existing + standard library classes which also extend :class:`Generic` + to support type variables inside ``[]``. + These types became redundant in Python 3.9 when the + corresponding pre-existing classes were enhanced to support ``[]``. - Type variable. + The redundant types are deprecated as of Python 3.9 but no + deprecation warnings will be issued by the interpreter. + It is expected that type checkers will flag the deprecated types + when the checked program targets Python 3.9 or newer. - Usage:: + The deprecated types will be removed from the :mod:`typing` module + in the first Python version released 5 years after the release of Python 3.9.0. + See details in :pep:`585`—*Type Hinting Generics In Standard Collections*. - T = TypeVar('T') # Can be anything - A = TypeVar('A', str, bytes) # Must be str or bytes - Type variables exist primarily for the benefit of static type - checkers. They serve as the parameters for generic types as well - as for generic function definitions. See class Generic for more - information on generic types. Generic functions work as follows:: +Special typing primitives +------------------------- - def repeat(x: T, n: int) -> Sequence[T]: - """Return a list containing n references to x.""" - return [x]*n +Special types +""""""""""""" - def longest(x: A, y: A) -> A: - """Return the longest of two strings.""" - return x if len(x) >= len(y) else y +These can be used as types in annotations and do not support ``[]``. - The latter example's signature is essentially the overloading - of ``(str, str) -> str`` and ``(bytes, bytes) -> bytes``. Also note - that if the arguments are instances of some subclass of :class:`str`, - the return type is still plain :class:`str`. +.. data:: Any - At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, - :func:`isinstance` and :func:`issubclass` should not be used with types. + Special type indicating an unconstrained type. - Type variables may be marked covariant or contravariant by passing - ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more - details. By default type variables are invariant. Alternatively, - a type variable may specify an upper bound using ``bound=<type>``. - This means that an actual type substituted (explicitly or implicitly) - for the type variable must be a subclass of the boundary type, - see :pep:`484`. + * Every type is compatible with :data:`Any`. + * :data:`Any` is compatible with every type. -.. class:: Generic +.. data:: NoReturn - Abstract base class for generic types. + Special type indicating that a function never returns. + For example:: - A generic type is typically declared by inheriting from an - instantiation of this class with one or more type variables. - For example, a generic mapping type might be defined as:: + from typing import NoReturn - class Mapping(Generic[KT, VT]): - def __getitem__(self, key: KT) -> VT: - ... - # Etc. + def stop() -> NoReturn: + raise RuntimeError('no way') - This class can then be used as follows:: + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 - X = TypeVar('X') - Y = TypeVar('Y') +Special forms +""""""""""""" - def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: - try: - return mapping[key] - except KeyError: - return default +These can be used as types in annotations using ``[]``, each having a unique syntax. -.. class:: Protocol(Generic) +.. data:: Tuple - Base class for protocol classes. Protocol classes are defined like this:: + Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items + with the first item of type X and the second of type Y. The type of + the empty tuple can be written as ``Tuple[()]``. - class Proto(Protocol): - def meth(self) -> int: - ... + Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding + to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple + of an int, a float and a string. - Such classes are primarily used with static type checkers that recognize - structural subtyping (static duck-typing), for example:: + To specify a variable-length tuple of homogeneous type, + use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` + is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. - class C: - def meth(self) -> int: - return 0 + .. deprecated:: 3.9 + :class:`builtins.tuple <tuple>` now supports ``[]``. See :pep:`585`. - def func(x: Proto) -> int: - return x.meth() +.. data:: Union - func(C()) # Passes static type check + Union type; ``Union[X, Y]`` means either X or Y. - See :pep:`544` for details. Protocol classes decorated with - :func:`runtime_checkable` (described later) act as simple-minded runtime - protocols that check only the presence of given attributes, ignoring their - type signatures. + To define a union, use e.g. ``Union[int, str]``. Details: - Protocol classes can be generic, for example:: + * The arguments must be types and there must be at least one. - class GenProto(Protocol[T]): - def meth(self) -> T: - ... + * Unions of unions are flattened, e.g.:: - .. versionadded:: 3.8 + Union[Union[int, str], float] == Union[int, str, float] + + * Unions of a single argument vanish, e.g.:: + + Union[int] == int # The constructor actually returns int + + * Redundant arguments are skipped, e.g.:: + + Union[int, str, int] == Union[int, str] + + * When comparing unions, the argument order is ignored, e.g.:: + + Union[int, str] == Union[str, int] + + * You cannot subclass or instantiate a union. + + * You cannot write ``Union[X][Y]``. + + * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. + + .. versionchanged:: 3.7 + Don't remove explicit subclasses from unions at runtime. + +.. data:: Optional + + Optional type. + + ``Optional[X]`` is equivalent to ``Union[X, None]``. + + Note that this is not the same concept as an optional argument, + which is one that has a default. An optional argument with a + default does not require the ``Optional`` qualifier on its type + annotation just because it is optional. For example:: + + def foo(arg: int = 0) -> None: + ... + + On the other hand, if an explicit value of ``None`` is allowed, the + use of ``Optional`` is appropriate, whether the argument is optional + or not. For example:: + + def foo(arg: Optional[int] = None) -> None: + ... + +.. data:: Callable + + Callable type; ``Callable[[int], str]`` is a function of (int) -> str. + + The subscription syntax must always be used with exactly two + values: the argument list and the return type. The argument list + must be a list of types or an ellipsis; the return type must be + a single type. + + There is no syntax to indicate optional or keyword arguments; + such function types are rarely used as callback types. + ``Callable[..., ReturnType]`` (literal ellipsis) can be used to + type hint a callable taking any number of arguments and returning + ``ReturnType``. A plain :data:`Callable` is equivalent to + ``Callable[..., Any]``, and in turn to + :class:`collections.abc.Callable`. + + .. deprecated:: 3.9 + :class:`collections.abc.Callable` now supports ``[]``. See :pep:`585`. .. class:: Type(Generic[CT_co]) @@ -577,374 +624,342 @@ The module defines the following classes, functions and decorators: .. versionadded:: 3.5.2 -.. class:: Iterable(Generic[T_co]) + .. deprecated:: 3.9 + :class:`builtins.type <type>` now supports ``[]``. See :pep:`585`. - A generic version of :class:`collections.abc.Iterable`. +.. data:: Literal -.. class:: Iterator(Iterable[T_co]) + A type that can be used to indicate to type checkers that the + corresponding variable or function parameter has a value equivalent to + the provided literal (or one of several literals). For example:: - A generic version of :class:`collections.abc.Iterator`. + def validate_simple(data: Any) -> Literal[True]: # always returns True + ... -.. class:: Reversible(Iterable[T_co]) + MODE = Literal['r', 'rb', 'w', 'wb'] + def open_helper(file: str, mode: MODE) -> str: + ... - A generic version of :class:`collections.abc.Reversible`. + open_helper('/some/path', 'r') # Passes type check + open_helper('/other/path', 'typo') # Error in type checker -.. class:: SupportsInt + ``Literal[...]`` cannot be subclassed. At runtime, an arbitrary value + is allowed as type argument to ``Literal[...]``, but type checkers may + impose restrictions. See :pep:`586` for more details about literal types. - An ABC with one abstract method ``__int__``. + .. versionadded:: 3.8 -.. class:: SupportsFloat +.. data:: ClassVar - An ABC with one abstract method ``__float__``. + Special type construct to mark class variables. -.. class:: SupportsComplex + As introduced in :pep:`526`, a variable annotation wrapped in ClassVar + indicates that a given attribute is intended to be used as a class variable + and should not be set on instances of that class. Usage:: - An ABC with one abstract method ``__complex__``. + class Starship: + stats: ClassVar[Dict[str, int]] = {} # class variable + damage: int = 10 # instance variable -.. class:: SupportsBytes + :data:`ClassVar` accepts only types and cannot be further subscribed. - An ABC with one abstract method ``__bytes__``. + :data:`ClassVar` is not a class itself, and should not + be used with :func:`isinstance` or :func:`issubclass`. + :data:`ClassVar` does not change Python runtime behavior, but + it can be used by third-party type checkers. For example, a type checker + might flag the following code as an error:: -.. class:: SupportsIndex + enterprise_d = Starship(3000) + enterprise_d.stats = {} # Error, setting class variable on instance + Starship.stats = {} # This is OK - An ABC with one abstract method ``__index__``. + .. versionadded:: 3.5.3 - .. versionadded:: 3.8 +.. data:: Final -.. class:: SupportsAbs + A special typing construct to indicate to type checkers that a name + cannot be re-assigned or overridden in a subclass. For example:: - An ABC with one abstract method ``__abs__`` that is covariant - in its return type. + MAX_SIZE: Final = 9000 + MAX_SIZE += 1 # Error reported by type checker -.. class:: SupportsRound + class Connection: + TIMEOUT: Final[int] = 10 - An ABC with one abstract method ``__round__`` - that is covariant in its return type. + class FastConnector(Connection): + TIMEOUT = 1 # Error reported by type checker -.. class:: Container(Generic[T_co]) + There is no runtime checking of these properties. See :pep:`591` for + more details. - A generic version of :class:`collections.abc.Container`. + .. versionadded:: 3.8 -.. class:: Hashable +.. data:: Annotated - An alias to :class:`collections.abc.Hashable` + A type, introduced in :pep:`593` (``Flexible function and variable + annotations``), to decorate existing types with context-specific metadata + (possibly multiple pieces of it, as ``Annotated`` is variadic). + Specifically, a type ``T`` can be annotated with metadata ``x`` via the + typehint ``Annotated[T, x]``. This metadata can be used for either static + analysis or at runtime. If a library (or tool) encounters a typehint + ``Annotated[T, x]`` and has no special logic for metadata ``x``, it + should ignore it and simply treat the type as ``T``. Unlike the + ``no_type_check`` functionality that currently exists in the ``typing`` + module which completely disables typechecking annotations on a function + or a class, the ``Annotated`` type allows for both static typechecking + of ``T`` (e.g., via mypy or Pyre, which can safely ignore ``x``) + together with runtime access to ``x`` within a specific application. -.. class:: Sized + Ultimately, the responsibility of how to interpret the annotations (if + at all) is the responsibility of the tool or library encountering the + ``Annotated`` type. A tool or library encountering an ``Annotated`` type + can scan through the annotations to determine if they are of interest + (e.g., using ``isinstance()``). - An alias to :class:`collections.abc.Sized` + When a tool or a library does not support annotations or encounters an + unknown annotation it should just ignore it and treat annotated type as + the underlying type. -.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) + It's up to the tool consuming the annotations to decide whether the + client is allowed to have several annotations on one type and how to + merge those annotations. - A generic version of :class:`collections.abc.Collection` + Since the ``Annotated`` type allows you to put several annotations of + the same (or different) type(s) on any node, the tools or libraries + consuming those annotations are in charge of dealing with potential + duplicates. For example, if you are doing value range analysis you might + allow this:: - .. versionadded:: 3.6.0 + T1 = Annotated[int, ValueRange(-10, 5)] + T2 = Annotated[T1, ValueRange(-20, 3)] -.. class:: AbstractSet(Sized, Collection[T_co]) + Passing ``include_extras=True`` to :func:`get_type_hints` lets one + access the extra annotations at runtime. - A generic version of :class:`collections.abc.Set`. + The details of the syntax: -.. class:: MutableSet(AbstractSet[T]) + * The first argument to ``Annotated`` must be a valid type - A generic version of :class:`collections.abc.MutableSet`. + * Multiple type annotations are supported (``Annotated`` supports variadic + arguments):: -.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) + Annotated[int, ValueRange(3, 10), ctype("char")] - A generic version of :class:`collections.abc.Mapping`. - This type can be used as follows:: + * ``Annotated`` must be called with at least two arguments ( + ``Annotated[int]`` is not valid) - def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: - return word_list[word] + * The order of the annotations is preserved and matters for equality + checks:: -.. class:: MutableMapping(Mapping[KT, VT]) + Annotated[int, ValueRange(3, 10), ctype("char")] != Annotated[ + int, ctype("char"), ValueRange(3, 10) + ] - A generic version of :class:`collections.abc.MutableMapping`. + * Nested ``Annotated`` types are flattened, with metadata ordered + starting with the innermost annotation:: -.. class:: Sequence(Reversible[T_co], Collection[T_co]) + Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[ + int, ValueRange(3, 10), ctype("char") + ] - A generic version of :class:`collections.abc.Sequence`. + * Duplicated annotations are not removed:: -.. class:: MutableSequence(Sequence[T]) + Annotated[int, ValueRange(3, 10)] != Annotated[ + int, ValueRange(3, 10), ValueRange(3, 10) + ] - A generic version of :class:`collections.abc.MutableSequence`. + * ``Annotated`` can be used with nested and generic aliases:: -.. class:: ByteString(Sequence[int]) + T = TypeVar('T') + Vec = Annotated[List[Tuple[T, T]], MaxLen(10)] + V = Vec[int] - A generic version of :class:`collections.abc.ByteString`. + V == Annotated[List[Tuple[int, int]], MaxLen(10)] - This type represents the types :class:`bytes`, :class:`bytearray`, - and :class:`memoryview` of byte sequences. + .. versionadded:: 3.9 - As a shorthand for this type, :class:`bytes` can be used to - annotate arguments of any of the types mentioned above. +Building generic types +"""""""""""""""""""""" -.. class:: Deque(deque, MutableSequence[T]) +These are not used in annotations. They are building blocks for creating generic types. - A generic version of :class:`collections.deque`. +.. class:: Generic - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 + Abstract base class for generic types. -.. class:: List(list, MutableSequence[T]) + A generic type is typically declared by inheriting from an + instantiation of this class with one or more type variables. + For example, a generic mapping type might be defined as:: - Generic version of :class:`list`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Sequence` or - :class:`Iterable`. + class Mapping(Generic[KT, VT]): + def __getitem__(self, key: KT) -> VT: + ... + # Etc. - This type may be used as follows:: + This class can then be used as follows:: - T = TypeVar('T', int, float) + X = TypeVar('X') + Y = TypeVar('Y') - def vec2(x: T, y: T) -> List[T]: - return [x, y] + def lookup_name(mapping: Mapping[X, Y], key: X, default: Y) -> Y: + try: + return mapping[key] + except KeyError: + return default - def keep_positives(vector: Sequence[T]) -> List[T]: - return [item for item in vector if item > 0] +.. class:: TypeVar -.. class:: Set(set, MutableSet[T]) + Type variable. - A generic version of :class:`builtins.set <set>`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`AbstractSet`. + Usage:: -.. class:: FrozenSet(frozenset, AbstractSet[T_co]) + T = TypeVar('T') # Can be anything + A = TypeVar('A', str, bytes) # Must be str or bytes - A generic version of :class:`builtins.frozenset <frozenset>`. + Type variables exist primarily for the benefit of static type + checkers. They serve as the parameters for generic types as well + as for generic function definitions. See class Generic for more + information on generic types. Generic functions work as follows:: -.. class:: MappingView(Sized, Iterable[T_co]) + def repeat(x: T, n: int) -> Sequence[T]: + """Return a list containing n references to x.""" + return [x]*n - A generic version of :class:`collections.abc.MappingView`. + def longest(x: A, y: A) -> A: + """Return the longest of two strings.""" + return x if len(x) >= len(y) else y -.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) + The latter example's signature is essentially the overloading + of ``(str, str) -> str`` and ``(bytes, bytes) -> bytes``. Also note + that if the arguments are instances of some subclass of :class:`str`, + the return type is still plain :class:`str`. - A generic version of :class:`collections.abc.KeysView`. + At runtime, ``isinstance(x, T)`` will raise :exc:`TypeError`. In general, + :func:`isinstance` and :func:`issubclass` should not be used with types. -.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) + Type variables may be marked covariant or contravariant by passing + ``covariant=True`` or ``contravariant=True``. See :pep:`484` for more + details. By default type variables are invariant. Alternatively, + a type variable may specify an upper bound using ``bound=<type>``. + This means that an actual type substituted (explicitly or implicitly) + for the type variable must be a subclass of the boundary type, + see :pep:`484`. - A generic version of :class:`collections.abc.ItemsView`. +.. data:: AnyStr -.. class:: ValuesView(MappingView[VT_co]) + ``AnyStr`` is a type variable defined as + ``AnyStr = TypeVar('AnyStr', str, bytes)``. - A generic version of :class:`collections.abc.ValuesView`. + It is meant to be used for functions that may accept any kind of string + without allowing different kinds of strings to mix. For example:: -.. class:: Awaitable(Generic[T_co]) + def concat(a: AnyStr, b: AnyStr) -> AnyStr: + return a + b - A generic version of :class:`collections.abc.Awaitable`. + concat(u"foo", u"bar") # Ok, output has type 'unicode' + concat(b"foo", b"bar") # Ok, output has type 'bytes' + concat(u"foo", b"bar") # Error, cannot mix unicode and bytes - .. versionadded:: 3.5.2 +.. class:: Protocol(Generic) -.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) + Base class for protocol classes. Protocol classes are defined like this:: - A generic version of :class:`collections.abc.Coroutine`. - The variance and order of type variables - correspond to those of :class:`Generator`, for example:: + class Proto(Protocol): + def meth(self) -> int: + ... - from typing import List, Coroutine - c = None # type: Coroutine[List[str], str, int] - ... - x = c.send('hi') # type: List[str] - async def bar() -> None: - x = await c # type: int + Such classes are primarily used with static type checkers that recognize + structural subtyping (static duck-typing), for example:: - .. versionadded:: 3.5.3 + class C: + def meth(self) -> int: + return 0 -.. class:: AsyncIterable(Generic[T_co]) + def func(x: Proto) -> int: + return x.meth() - A generic version of :class:`collections.abc.AsyncIterable`. + func(C()) # Passes static type check - .. versionadded:: 3.5.2 + See :pep:`544` for details. Protocol classes decorated with + :func:`runtime_checkable` (described later) act as simple-minded runtime + protocols that check only the presence of given attributes, ignoring their + type signatures. -.. class:: AsyncIterator(AsyncIterable[T_co]) + Protocol classes can be generic, for example:: - A generic version of :class:`collections.abc.AsyncIterator`. + class GenProto(Protocol[T]): + def meth(self) -> T: + ... - .. versionadded:: 3.5.2 + .. versionadded:: 3.8 -.. class:: ContextManager(Generic[T_co]) +.. decorator:: runtime_checkable - A generic version of :class:`contextlib.AbstractContextManager`. + Mark a protocol class as a runtime protocol. - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.0 + Such a protocol can be used with :func:`isinstance` and :func:`issubclass`. + This raises :exc:`TypeError` when applied to a non-protocol class. This + allows a simple-minded structural check, very similar to "one trick ponies" + in :mod:`collections.abc` such as :class:`Iterable`. For example:: -.. class:: AsyncContextManager(Generic[T_co]) + @runtime_checkable + class Closable(Protocol): + def close(self): ... - A generic version of :class:`contextlib.AbstractAsyncContextManager`. + assert isinstance(open('/some/file'), Closable) - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 + .. note:: -.. class:: Dict(dict, MutableMapping[KT, VT]) + :func:`runtime_checkable` will check only the presence of the required methods, + not their type signatures! For example, :class:`builtins.complex <complex>` + implements :func:`__float__`, therefore it passes an :func:`issubclass` check + against :class:`SupportsFloat`. However, the ``complex.__float__`` method + exists only to raise a :class:`TypeError` with a more informative message. - A generic version of :class:`dict`. - Useful for annotating return types. To annotate arguments it is preferred - to use an abstract collection type such as :class:`Mapping`. + .. versionadded:: 3.8 - This type can be used as follows:: +Other special directives +"""""""""""""""""""""""" - def count_words(text: str) -> Dict[str, int]: - ... +These are not used in annotations. They are building blocks for declaring types. -.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) +.. class:: NamedTuple - A generic version of :class:`collections.defaultdict`. + Typed version of :func:`collections.namedtuple`. - .. versionadded:: 3.5.2 + Usage:: -.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) + class Employee(NamedTuple): + name: str + id: int - A generic version of :class:`collections.OrderedDict`. + This is equivalent to:: - .. versionadded:: 3.7.2 + Employee = collections.namedtuple('Employee', ['name', 'id']) -.. class:: Counter(collections.Counter, Dict[T, int]) + To give a field a default value, you can assign to it in the class body:: - A generic version of :class:`collections.Counter`. + class Employee(NamedTuple): + name: str + id: int = 3 - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 + employee = Employee('Guido') + assert employee.id == 3 -.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) + Fields with a default value must come after any fields without a default. - A generic version of :class:`collections.ChainMap`. + The resulting class has an extra attribute ``__annotations__`` giving a + dict that maps the field names to the field types. (The field names are in + the ``_fields`` attribute and the default values are in the + ``_field_defaults`` attribute both of which are part of the namedtuple + API.) - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.1 + ``NamedTuple`` subclasses can also have docstrings and methods:: -.. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) - - A generator can be annotated by the generic type - ``Generator[YieldType, SendType, ReturnType]``. For example:: - - def echo_round() -> Generator[int, float, str]: - sent = yield 0 - while sent >= 0: - sent = yield round(sent) - return 'Done' - - Note that unlike many other generics in the typing module, the ``SendType`` - of :class:`Generator` behaves contravariantly, not covariantly or - invariantly. - - If your generator will only yield values, set the ``SendType`` and - ``ReturnType`` to ``None``:: - - def infinite_stream(start: int) -> Generator[int, None, None]: - while True: - yield start - start += 1 - - Alternatively, annotate your generator as having a return type of - either ``Iterable[YieldType]`` or ``Iterator[YieldType]``:: - - def infinite_stream(start: int) -> Iterator[int]: - while True: - yield start - start += 1 - -.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) - - An async generator can be annotated by the generic type - ``AsyncGenerator[YieldType, SendType]``. For example:: - - async def echo_round() -> AsyncGenerator[int, float]: - sent = yield 0 - while sent >= 0.0: - rounded = await round(sent) - sent = yield rounded - - Unlike normal generators, async generators cannot return a value, so there - is no ``ReturnType`` type parameter. As with :class:`Generator`, the - ``SendType`` behaves contravariantly. - - If your generator will only yield values, set the ``SendType`` to - ``None``:: - - async def infinite_stream(start: int) -> AsyncGenerator[int, None]: - while True: - yield start - start = await increment(start) - - Alternatively, annotate your generator as having a return type of - either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: - - async def infinite_stream(start: int) -> AsyncIterator[int]: - while True: - yield start - start = await increment(start) - - .. versionadded:: 3.6.1 - -.. class:: Text - - ``Text`` is an alias for ``str``. It is provided to supply a forward - compatible path for Python 2 code: in Python 2, ``Text`` is an alias for - ``unicode``. - - Use ``Text`` to indicate that a value must contain a unicode string in - a manner that is compatible with both Python 2 and Python 3:: - - def add_unicode_checkmark(text: Text) -> Text: - return text + u' \u2713' - - .. versionadded:: 3.5.2 - -.. class:: IO - TextIO - BinaryIO - - Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` - and ``BinaryIO(IO[bytes])`` - represent the types of I/O streams such as returned by - :func:`open`. - -.. class:: Pattern - Match - - These type aliases - correspond to the return types from :func:`re.compile` and - :func:`re.match`. These types (and the corresponding functions) - are generic in ``AnyStr`` and can be made specific by writing - ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or - ``Match[bytes]``. - -.. class:: NamedTuple - - Typed version of :func:`collections.namedtuple`. - - Usage:: - - class Employee(NamedTuple): - name: str - id: int - - This is equivalent to:: - - Employee = collections.namedtuple('Employee', ['name', 'id']) - - To give a field a default value, you can assign to it in the class body:: - - class Employee(NamedTuple): - name: str - id: int = 3 - - employee = Employee('Guido') - assert employee.id == 3 - - Fields with a default value must come after any fields without a default. - - The resulting class has an extra attribute ``__annotations__`` giving a - dict that maps the field names to the field types. (The field names are in - the ``_fields`` attribute and the default values are in the - ``_field_defaults`` attribute both of which are part of the namedtuple - API.) - - ``NamedTuple`` subclasses can also have docstrings and methods:: - - class Employee(NamedTuple): - """Represents an employee.""" - name: str - id: int = 3 + class Employee(NamedTuple): + """Represents an employee.""" + name: str + id: int = 3 def __repr__(self) -> str: return f'<Employee {self.name}, id={self.id}>' @@ -967,13 +982,23 @@ The module defines the following classes, functions and decorators: Removed the ``_field_types`` attribute in favor of the more standard ``__annotations__`` attribute which has the same information. +.. function:: NewType(name, tp) + + A helper function to indicate a distinct type to a typechecker, + see :ref:`distinct`. At runtime it returns a function that returns + its argument. Usage:: + + UserId = NewType('UserId', int) + first_user = UserId(1) + + .. versionadded:: 3.5.2 .. class:: TypedDict(dict) - A simple typed namespace. At runtime it is equivalent to - a plain :class:`dict`. + Special construct to add type hints to a dictionary. + At runtime it is a plain :class:`dict`. - ``TypedDict`` creates a dictionary type that expects all of its + ``TypedDict`` declares a dictionary type that expects all of its instances to have a certain set of keys, where each key is associated with a value of a consistent type. This expectation is not checked at runtime but is only enforced by type checkers. @@ -1014,474 +1039,658 @@ The module defines the following classes, functions and decorators: .. versionadded:: 3.8 -.. class:: ForwardRef +Generic concrete collections +---------------------------- - A class used for internal typing representation of string forward references. - For example, ``List["SomeClass"]`` is implicitly transformed into - ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by - a user, but may be used by introspection tools. +Corresponding to built-in types +""""""""""""""""""""""""""""""" -.. function:: NewType(name, tp) +.. class:: Dict(dict, MutableMapping[KT, VT]) - A helper function to indicate a distinct type to a typechecker, - see :ref:`distinct`. At runtime it returns a function that returns - its argument. Usage:: + A generic version of :class:`dict`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Mapping`. - UserId = NewType('UserId', int) - first_user = UserId(1) + This type can be used as follows:: - .. versionadded:: 3.5.2 + def count_words(text: str) -> Dict[str, int]: + ... -.. function:: cast(typ, val) + .. deprecated:: 3.9 + :class:`builtins.dict <dict>` now supports ``[]``. See :pep:`585`. - Cast a value to a type. +.. class:: List(list, MutableSequence[T]) - This returns the value unchanged. To the type checker this - signals that the return value has the designated type, but at - runtime we intentionally don't check anything (we want this - to be as fast as possible). + Generic version of :class:`list`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`Sequence` or + :class:`Iterable`. -.. function:: get_type_hints(obj, globalns=None, localns=None, include_extras=False) + This type may be used as follows:: - Return a dictionary containing type hints for a function, method, module - or class object. + T = TypeVar('T', int, float) - This is often the same as ``obj.__annotations__``. In addition, - forward references encoded as string literals are handled by evaluating - them in ``globals`` and ``locals`` namespaces. If necessary, - ``Optional[t]`` is added for function and method annotations if a default - value equal to ``None`` is set. For a class ``C``, return - a dictionary constructed by merging all the ``__annotations__`` along - ``C.__mro__`` in reverse order. + def vec2(x: T, y: T) -> List[T]: + return [x, y] - The function recursively replaces all ``Annotated[T, ...]`` with ``T``, - unless ``include_extras`` is set to ``True`` (see :class:`Annotated` for - more information). For example:: + def keep_positives(vector: Sequence[T]) -> List[T]: + return [item for item in vector if item > 0] - class Student(NamedTuple): - name: Annotated[str, 'some marker'] + .. deprecated:: 3.9 + :class:`builtins.list <list>` now supports ``[]``. See :pep:`585`. - get_type_hints(Student) == {'name': str} - get_type_hints(Student, include_extras=False) == {'name': str} - get_type_hints(Student, include_extras=True) == { - 'name': Annotated[str, 'some marker'] - } +.. class:: Set(set, MutableSet[T]) - .. versionchanged:: 3.9 - Added ``include_extras`` parameter as part of :pep:`593`. + A generic version of :class:`builtins.set <set>`. + Useful for annotating return types. To annotate arguments it is preferred + to use an abstract collection type such as :class:`AbstractSet`. -.. function:: get_origin(tp) -.. function:: get_args(tp) + .. deprecated:: 3.9 + :class:`builtins.set <set>` now supports ``[]``. See :pep:`585`. - Provide basic introspection for generic types and special typing forms. +.. class:: FrozenSet(frozenset, AbstractSet[T_co]) - For a typing object of the form ``X[Y, Z, ...]`` these functions return - ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or - :mod:`collections` class, it gets normalized to the original class. - For unsupported objects return ``None`` and ``()`` correspondingly. - Examples:: + A generic version of :class:`builtins.frozenset <frozenset>`. - assert get_origin(Dict[str, int]) is dict - assert get_args(Dict[int, str]) == (int, str) + .. deprecated:: 3.9 + :class:`builtins.frozenset <frozenset>` now supports ``[]``. See :pep:`585`. - assert get_origin(Union[int, str]) is Union - assert get_args(Union[int, str]) == (int, str) +.. note:: :data:`Tuple` is a special form. - .. versionadded:: 3.8 +Corresponding to types in :mod:`collections` +"""""""""""""""""""""""""""""""""""""""""""" -.. decorator:: overload +.. class:: DefaultDict(collections.defaultdict, MutableMapping[KT, VT]) - The ``@overload`` decorator allows describing functions and methods - that support multiple different combinations of argument types. A series - of ``@overload``-decorated definitions must be followed by exactly one - non-``@overload``-decorated definition (for the same function/method). - The ``@overload``-decorated definitions are for the benefit of the - type checker only, since they will be overwritten by the - non-``@overload``-decorated definition, while the latter is used at - runtime but should be ignored by a type checker. At runtime, calling - a ``@overload``-decorated function directly will raise - :exc:`NotImplementedError`. An example of overload that gives a more - precise type than can be expressed using a union or a type variable:: + A generic version of :class:`collections.defaultdict`. - @overload - def process(response: None) -> None: - ... - @overload - def process(response: int) -> Tuple[int, str]: - ... - @overload - def process(response: bytes) -> str: - ... - def process(response): - <actual implementation> + .. versionadded:: 3.5.2 - See :pep:`484` for details and comparison with other typing semantics. + .. deprecated:: 3.9 + :class:`collections.defaultdict` now supports ``[]``. See :pep:`585`. -.. decorator:: final +.. class:: OrderedDict(collections.OrderedDict, MutableMapping[KT, VT]) - A decorator to indicate to type checkers that the decorated method - cannot be overridden, and the decorated class cannot be subclassed. - For example:: + A generic version of :class:`collections.OrderedDict`. - class Base: - @final - def done(self) -> None: - ... - class Sub(Base): - def done(self) -> None: # Error reported by type checker - ... + .. versionadded:: 3.7.2 - @final - class Leaf: - ... - class Other(Leaf): # Error reported by type checker - ... + .. deprecated:: 3.9 + :class:`collections.OrderedDict` now supports ``[]``. See :pep:`585`. - There is no runtime checking of these properties. See :pep:`591` for - more details. +.. class:: ChainMap(collections.ChainMap, MutableMapping[KT, VT]) - .. versionadded:: 3.8 + A generic version of :class:`collections.ChainMap`. -.. decorator:: no_type_check + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 - Decorator to indicate that annotations are not type hints. + .. deprecated:: 3.9 + :class:`collections.ChainMap` now supports ``[]``. See :pep:`585`. - This works as class or function :term:`decorator`. With a class, it - applies recursively to all methods defined in that class (but not - to methods defined in its superclasses or subclasses). +.. class:: Counter(collections.Counter, Dict[T, int]) - This mutates the function(s) in place. + A generic version of :class:`collections.Counter`. -.. decorator:: no_type_check_decorator + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 - Decorator to give another decorator the :func:`no_type_check` effect. + .. deprecated:: 3.9 + :class:`collections.Counter` now supports ``[]``. See :pep:`585`. - This wraps the decorator with something that wraps the decorated - function in :func:`no_type_check`. +.. class:: Deque(deque, MutableSequence[T]) -.. decorator:: type_check_only + A generic version of :class:`collections.deque`. - Decorator to mark a class or function to be unavailable at runtime. + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.1 - This decorator is itself not available at runtime. It is mainly - intended to mark classes that are defined in type stub files if - an implementation returns an instance of a private class:: + .. deprecated:: 3.9 + :class:`collections.deque` now supports ``[]``. See :pep:`585`. - @type_check_only - class Response: # private or not available at runtime - code: int - def get_header(self, name: str) -> str: ... +Other concrete types +"""""""""""""""""""" - def fetch_response() -> Response: ... +.. class:: IO + TextIO + BinaryIO - Note that returning instances of private classes is not recommended. - It is usually preferable to make such classes public. + Generic type ``IO[AnyStr]`` and its subclasses ``TextIO(IO[str])`` + and ``BinaryIO(IO[bytes])`` + represent the types of I/O streams such as returned by + :func:`open`. These types are also in the ``typing.io`` namespace. -.. decorator:: runtime_checkable +.. class:: Pattern + Match - Mark a protocol class as a runtime protocol. + These type aliases + correspond to the return types from :func:`re.compile` and + :func:`re.match`. These types (and the corresponding functions) + are generic in ``AnyStr`` and can be made specific by writing + ``Pattern[str]``, ``Pattern[bytes]``, ``Match[str]``, or + ``Match[bytes]``. These types are also in the ``typing.re`` namespace. - Such a protocol can be used with :func:`isinstance` and :func:`issubclass`. - This raises :exc:`TypeError` when applied to a non-protocol class. This - allows a simple-minded structural check, very similar to "one trick ponies" - in :mod:`collections.abc` such as :class:`Iterable`. For example:: + .. deprecated:: 3.9 + Classes ``Pattern`` and ``Match`` from :mod:`re` now support ``[]``. See :pep:`585`. - @runtime_checkable - class Closable(Protocol): - def close(self): ... +.. class:: Text - assert isinstance(open('/some/file'), Closable) + ``Text`` is an alias for ``str``. It is provided to supply a forward + compatible path for Python 2 code: in Python 2, ``Text`` is an alias for + ``unicode``. - **Warning:** this will check only the presence of the required methods, - not their type signatures! + Use ``Text`` to indicate that a value must contain a unicode string in + a manner that is compatible with both Python 2 and Python 3:: - .. versionadded:: 3.8 + def add_unicode_checkmark(text: Text) -> Text: + return text + u' \u2713' -.. data:: Any + .. versionadded:: 3.5.2 - Special type indicating an unconstrained type. +Abstract Base Classes +--------------------- - * Every type is compatible with :data:`Any`. - * :data:`Any` is compatible with every type. +Corresponding to collections in :mod:`collections.abc` +"""""""""""""""""""""""""""""""""""""""""""""""""""""" -.. data:: NoReturn +.. class:: AbstractSet(Sized, Collection[T_co]) - Special type indicating that a function never returns. - For example:: + A generic version of :class:`collections.abc.Set`. - from typing import NoReturn + .. deprecated:: 3.9 + :class:`collections.abc.Set` now supports ``[]``. See :pep:`585`. - def stop() -> NoReturn: - raise RuntimeError('no way') +.. class:: ByteString(Sequence[int]) - .. versionadded:: 3.5.4 - .. versionadded:: 3.6.2 + A generic version of :class:`collections.abc.ByteString`. -.. data:: Union + This type represents the types :class:`bytes`, :class:`bytearray`, + and :class:`memoryview` of byte sequences. - Union type; ``Union[X, Y]`` means either X or Y. + As a shorthand for this type, :class:`bytes` can be used to + annotate arguments of any of the types mentioned above. - To define a union, use e.g. ``Union[int, str]``. Details: + .. deprecated:: 3.9 + :class:`collections.abc.ByteString` now supports ``[]``. See :pep:`585`. - * The arguments must be types and there must be at least one. +.. class:: Collection(Sized, Iterable[T_co], Container[T_co]) - * Unions of unions are flattened, e.g.:: + A generic version of :class:`collections.abc.Collection` - Union[Union[int, str], float] == Union[int, str, float] + .. versionadded:: 3.6.0 - * Unions of a single argument vanish, e.g.:: + .. deprecated:: 3.9 + :class:`collections.abc.Collection` now supports ``[]``. See :pep:`585`. - Union[int] == int # The constructor actually returns int +.. class:: Container(Generic[T_co]) - * Redundant arguments are skipped, e.g.:: + A generic version of :class:`collections.abc.Container`. - Union[int, str, int] == Union[int, str] + .. deprecated:: 3.9 + :class:`collections.abc.Container` now supports ``[]``. See :pep:`585`. - * When comparing unions, the argument order is ignored, e.g.:: +.. class:: ItemsView(MappingView, Generic[KT_co, VT_co]) - Union[int, str] == Union[str, int] + A generic version of :class:`collections.abc.ItemsView`. - * You cannot subclass or instantiate a union. + .. deprecated:: 3.9 + :class:`collections.abc.ItemsView` now supports ``[]``. See :pep:`585`. - * You cannot write ``Union[X][Y]``. +.. class:: KeysView(MappingView[KT_co], AbstractSet[KT_co]) - * You can use ``Optional[X]`` as a shorthand for ``Union[X, None]``. + A generic version of :class:`collections.abc.KeysView`. - .. versionchanged:: 3.7 - Don't remove explicit subclasses from unions at runtime. + .. deprecated:: 3.9 + :class:`collections.abc.KeysView` now supports ``[]``. See :pep:`585`. -.. data:: Optional +.. class:: Mapping(Sized, Collection[KT], Generic[VT_co]) - Optional type. + A generic version of :class:`collections.abc.Mapping`. + This type can be used as follows:: - ``Optional[X]`` is equivalent to ``Union[X, None]``. + def get_position_in_index(word_list: Mapping[str, int], word: str) -> int: + return word_list[word] - Note that this is not the same concept as an optional argument, - which is one that has a default. An optional argument with a - default does not require the ``Optional`` qualifier on its type - annotation just because it is optional. For example:: + .. deprecated:: 3.9 + :class:`collections.abc.Mapping` now supports ``[]``. See :pep:`585`. - def foo(arg: int = 0) -> None: - ... +.. class:: MappingView(Sized, Iterable[T_co]) - On the other hand, if an explicit value of ``None`` is allowed, the - use of ``Optional`` is appropriate, whether the argument is optional - or not. For example:: + A generic version of :class:`collections.abc.MappingView`. - def foo(arg: Optional[int] = None) -> None: - ... + .. deprecated:: 3.9 + :class:`collections.abc.MappingView` now supports ``[]``. See :pep:`585`. -.. data:: Tuple +.. class:: MutableMapping(Mapping[KT, VT]) - Tuple type; ``Tuple[X, Y]`` is the type of a tuple of two items - with the first item of type X and the second of type Y. The type of - the empty tuple can be written as ``Tuple[()]``. + A generic version of :class:`collections.abc.MutableMapping`. - Example: ``Tuple[T1, T2]`` is a tuple of two elements corresponding - to type variables T1 and T2. ``Tuple[int, float, str]`` is a tuple - of an int, a float and a string. + .. deprecated:: 3.9 + :class:`collections.abc.MutableMapping` now supports ``[]``. See :pep:`585`. - To specify a variable-length tuple of homogeneous type, - use literal ellipsis, e.g. ``Tuple[int, ...]``. A plain :data:`Tuple` - is equivalent to ``Tuple[Any, ...]``, and in turn to :class:`tuple`. +.. class:: MutableSequence(Sequence[T]) -.. data:: Callable + A generic version of :class:`collections.abc.MutableSequence`. - Callable type; ``Callable[[int], str]`` is a function of (int) -> str. + .. deprecated:: 3.9 + :class:`collections.abc.MutableSequence` now supports ``[]``. See :pep:`585`. - The subscription syntax must always be used with exactly two - values: the argument list and the return type. The argument list - must be a list of types or an ellipsis; the return type must be - a single type. +.. class:: MutableSet(AbstractSet[T]) - There is no syntax to indicate optional or keyword arguments; - such function types are rarely used as callback types. - ``Callable[..., ReturnType]`` (literal ellipsis) can be used to - type hint a callable taking any number of arguments and returning - ``ReturnType``. A plain :data:`Callable` is equivalent to - ``Callable[..., Any]``, and in turn to - :class:`collections.abc.Callable`. + A generic version of :class:`collections.abc.MutableSet`. -.. data:: Literal + .. deprecated:: 3.9 + :class:`collections.abc.MutableSet` now supports ``[]``. See :pep:`585`. - A type that can be used to indicate to type checkers that the - corresponding variable or function parameter has a value equivalent to - the provided literal (or one of several literals). For example:: +.. class:: Sequence(Reversible[T_co], Collection[T_co]) - def validate_simple(data: Any) -> Literal[True]: # always returns True - ... + A generic version of :class:`collections.abc.Sequence`. - MODE = Literal['r', 'rb', 'w', 'wb'] - def open_helper(file: str, mode: MODE) -> str: - ... + .. deprecated:: 3.9 + :class:`collections.abc.Sequence` now supports ``[]``. See :pep:`585`. - open_helper('/some/path', 'r') # Passes type check - open_helper('/other/path', 'typo') # Error in type checker +.. class:: ValuesView(MappingView[VT_co]) - ``Literal[...]`` cannot be subclassed. At runtime, an arbitrary value - is allowed as type argument to ``Literal[...]``, but type checkers may - impose restrictions. See :pep:`586` for more details about literal types. + A generic version of :class:`collections.abc.ValuesView`. - .. versionadded:: 3.8 + .. deprecated:: 3.9 + :class:`collections.abc.ValuesView` now supports ``[]``. See :pep:`585`. -.. data:: ClassVar +Corresponding to other types in :mod:`collections.abc` +"""""""""""""""""""""""""""""""""""""""""""""""""""""" - Special type construct to mark class variables. +.. class:: Iterable(Generic[T_co]) - As introduced in :pep:`526`, a variable annotation wrapped in ClassVar - indicates that a given attribute is intended to be used as a class variable - and should not be set on instances of that class. Usage:: + A generic version of :class:`collections.abc.Iterable`. - class Starship: - stats: ClassVar[Dict[str, int]] = {} # class variable - damage: int = 10 # instance variable + .. deprecated:: 3.9 + :class:`collections.abc.Iterable` now supports ``[]``. See :pep:`585`. - :data:`ClassVar` accepts only types and cannot be further subscribed. +.. class:: Iterator(Iterable[T_co]) - :data:`ClassVar` is not a class itself, and should not - be used with :func:`isinstance` or :func:`issubclass`. - :data:`ClassVar` does not change Python runtime behavior, but - it can be used by third-party type checkers. For example, a type checker - might flag the following code as an error:: + A generic version of :class:`collections.abc.Iterator`. - enterprise_d = Starship(3000) - enterprise_d.stats = {} # Error, setting class variable on instance - Starship.stats = {} # This is OK + .. deprecated:: 3.9 + :class:`collections.abc.Iterator` now supports ``[]``. See :pep:`585`. - .. versionadded:: 3.5.3 +.. class:: Generator(Iterator[T_co], Generic[T_co, T_contra, V_co]) -.. data:: Final + A generator can be annotated by the generic type + ``Generator[YieldType, SendType, ReturnType]``. For example:: - A special typing construct to indicate to type checkers that a name - cannot be re-assigned or overridden in a subclass. For example:: + def echo_round() -> Generator[int, float, str]: + sent = yield 0 + while sent >= 0: + sent = yield round(sent) + return 'Done' - MAX_SIZE: Final = 9000 - MAX_SIZE += 1 # Error reported by type checker + Note that unlike many other generics in the typing module, the ``SendType`` + of :class:`Generator` behaves contravariantly, not covariantly or + invariantly. - class Connection: - TIMEOUT: Final[int] = 10 + If your generator will only yield values, set the ``SendType`` and + ``ReturnType`` to ``None``:: - class FastConnector(Connection): - TIMEOUT = 1 # Error reported by type checker + def infinite_stream(start: int) -> Generator[int, None, None]: + while True: + yield start + start += 1 - There is no runtime checking of these properties. See :pep:`591` for - more details. + Alternatively, annotate your generator as having a return type of + either ``Iterable[YieldType]`` or ``Iterator[YieldType]``:: - .. versionadded:: 3.8 + def infinite_stream(start: int) -> Iterator[int]: + while True: + yield start + start += 1 -.. data:: AnyStr + .. deprecated:: 3.9 + :class:`collections.abc.Generator` now supports ``[]``. See :pep:`585`. - ``AnyStr`` is a type variable defined as - ``AnyStr = TypeVar('AnyStr', str, bytes)``. +.. class:: Hashable - It is meant to be used for functions that may accept any kind of string - without allowing different kinds of strings to mix. For example:: + An alias to :class:`collections.abc.Hashable` - def concat(a: AnyStr, b: AnyStr) -> AnyStr: - return a + b +.. class:: Reversible(Iterable[T_co]) - concat(u"foo", u"bar") # Ok, output has type 'unicode' - concat(b"foo", b"bar") # Ok, output has type 'bytes' - concat(u"foo", b"bar") # Error, cannot mix unicode and bytes + A generic version of :class:`collections.abc.Reversible`. -.. data:: TYPE_CHECKING + .. deprecated:: 3.9 + :class:`collections.abc.Reversible` now supports ``[]``. See :pep:`585`. - A special constant that is assumed to be ``True`` by 3rd party static - type checkers. It is ``False`` at runtime. Usage:: +.. class:: Sized - if TYPE_CHECKING: - import expensive_mod + An alias to :class:`collections.abc.Sized` - def fun(arg: 'expensive_mod.SomeType') -> None: - local_var: expensive_mod.AnotherType = other_fun() +Asynchronous programming +"""""""""""""""""""""""" - Note that the first type annotation must be enclosed in quotes, making it a - "forward reference", to hide the ``expensive_mod`` reference from the - interpreter runtime. Type annotations for local variables are not - evaluated, so the second annotation does not need to be enclosed in quotes. +.. class:: Coroutine(Awaitable[V_co], Generic[T_co T_contra, V_co]) + + A generic version of :class:`collections.abc.Coroutine`. + The variance and order of type variables + correspond to those of :class:`Generator`, for example:: + + from typing import List, Coroutine + c = None # type: Coroutine[List[str], str, int] + ... + x = c.send('hi') # type: List[str] + async def bar() -> None: + x = await c # type: int + + .. versionadded:: 3.5.3 + + .. deprecated:: 3.9 + :class:`collections.abc.Coroutine` now supports ``[]``. See :pep:`585`. + +.. class:: AsyncGenerator(AsyncIterator[T_co], Generic[T_co, T_contra]) + + An async generator can be annotated by the generic type + ``AsyncGenerator[YieldType, SendType]``. For example:: + + async def echo_round() -> AsyncGenerator[int, float]: + sent = yield 0 + while sent >= 0.0: + rounded = await round(sent) + sent = yield rounded + + Unlike normal generators, async generators cannot return a value, so there + is no ``ReturnType`` type parameter. As with :class:`Generator`, the + ``SendType`` behaves contravariantly. + + If your generator will only yield values, set the ``SendType`` to + ``None``:: + + async def infinite_stream(start: int) -> AsyncGenerator[int, None]: + while True: + yield start + start = await increment(start) + + Alternatively, annotate your generator as having a return type of + either ``AsyncIterable[YieldType]`` or ``AsyncIterator[YieldType]``:: + + async def infinite_stream(start: int) -> AsyncIterator[int]: + while True: + yield start + start = await increment(start) + + .. versionadded:: 3.6.1 + + .. deprecated:: 3.9 + :class:`collections.abc.AsyncGenerator` now supports ``[]``. See :pep:`585`. + +.. class:: AsyncIterable(Generic[T_co]) + + A generic version of :class:`collections.abc.AsyncIterable`. .. versionadded:: 3.5.2 -.. data:: Annotated + .. deprecated:: 3.9 + :class:`collections.abc.AsyncIterable` now supports ``[]``. See :pep:`585`. - A type, introduced in :pep:`593` (``Flexible function and variable - annotations``), to decorate existing types with context-specific metadata - (possibly multiple pieces of it, as ``Annotated`` is variadic). - Specifically, a type ``T`` can be annotated with metadata ``x`` via the - typehint ``Annotated[T, x]``. This metadata can be used for either static - analysis or at runtime. If a library (or tool) encounters a typehint - ``Annotated[T, x]`` and has no special logic for metadata ``x``, it - should ignore it and simply treat the type as ``T``. Unlike the - ``no_type_check`` functionality that currently exists in the ``typing`` - module which completely disables typechecking annotations on a function - or a class, the ``Annotated`` type allows for both static typechecking - of ``T`` (e.g., via mypy or Pyre, which can safely ignore ``x``) - together with runtime access to ``x`` within a specific application. +.. class:: AsyncIterator(AsyncIterable[T_co]) - Ultimately, the responsibility of how to interpret the annotations (if - at all) is the responsibility of the tool or library encountering the - ``Annotated`` type. A tool or library encountering an ``Annotated`` type - can scan through the annotations to determine if they are of interest - (e.g., using ``isinstance()``). + A generic version of :class:`collections.abc.AsyncIterator`. - When a tool or a library does not support annotations or encounters an - unknown annotation it should just ignore it and treat annotated type as - the underlying type. + .. versionadded:: 3.5.2 - It's up to the tool consuming the annotations to decide whether the - client is allowed to have several annotations on one type and how to - merge those annotations. + .. deprecated:: 3.9 + :class:`collections.abc.AsyncIterator` now supports ``[]``. See :pep:`585`. - Since the ``Annotated`` type allows you to put several annotations of - the same (or different) type(s) on any node, the tools or libraries - consuming those annotations are in charge of dealing with potential - duplicates. For example, if you are doing value range analysis you might - allow this:: +.. class:: Awaitable(Generic[T_co]) - T1 = Annotated[int, ValueRange(-10, 5)] - T2 = Annotated[T1, ValueRange(-20, 3)] + A generic version of :class:`collections.abc.Awaitable`. - Passing ``include_extras=True`` to :func:`get_type_hints` lets one - access the extra annotations at runtime. + .. versionadded:: 3.5.2 - The details of the syntax: + .. deprecated:: 3.9 + :class:`collections.abc.Awaitable` now supports ``[]``. See :pep:`585`. - * The first argument to ``Annotated`` must be a valid type - * Multiple type annotations are supported (``Annotated`` supports variadic - arguments):: +Context manager types +""""""""""""""""""""" - Annotated[int, ValueRange(3, 10), ctype("char")] +.. class:: ContextManager(Generic[T_co]) - * ``Annotated`` must be called with at least two arguments ( - ``Annotated[int]`` is not valid) + A generic version of :class:`contextlib.AbstractContextManager`. - * The order of the annotations is preserved and matters for equality - checks:: + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.0 - Annotated[int, ValueRange(3, 10), ctype("char")] != Annotated[ - int, ctype("char"), ValueRange(3, 10) - ] + .. deprecated:: 3.9 + :class:`collections.contextlib.AbstractContextManager` now supports ``[]``. See :pep:`585`. - * Nested ``Annotated`` types are flattened, with metadata ordered - starting with the innermost annotation:: +.. class:: AsyncContextManager(Generic[T_co]) - Annotated[Annotated[int, ValueRange(3, 10)], ctype("char")] == Annotated[ - int, ValueRange(3, 10), ctype("char") - ] + A generic version of :class:`contextlib.AbstractAsyncContextManager`. - * Duplicated annotations are not removed:: + .. versionadded:: 3.5.4 + .. versionadded:: 3.6.2 - Annotated[int, ValueRange(3, 10)] != Annotated[ - int, ValueRange(3, 10), ValueRange(3, 10) - ] + .. deprecated:: 3.9 + :class:`collections.contextlib.AbstractAsyncContextManager` now supports ``[]``. See :pep:`585`. - * ``Annotated`` can be used with nested and generic aliases:: +Protocols +--------- - T = TypeVar('T') - Vec = Annotated[List[Tuple[T, T]], MaxLen(10)] - V = Vec[int] +These protocols are decorated with :func:`runtime_checkable`. - V == Annotated[List[Tuple[int, int]], MaxLen(10)] +.. class:: SupportsAbs + + An ABC with one abstract method ``__abs__`` that is covariant + in its return type. + +.. class:: SupportsBytes + + An ABC with one abstract method ``__bytes__``. + +.. class:: SupportsComplex + + An ABC with one abstract method ``__complex__``. + +.. class:: SupportsFloat + + An ABC with one abstract method ``__float__``. + +.. class:: SupportsIndex + + An ABC with one abstract method ``__index__``. + + .. versionadded:: 3.8 + +.. class:: SupportsInt + + An ABC with one abstract method ``__int__``. + +.. class:: SupportsRound + + An ABC with one abstract method ``__round__`` + that is covariant in its return type. + +Functions and decorators +------------------------ + +.. function:: cast(typ, val) + + Cast a value to a type. + + This returns the value unchanged. To the type checker this + signals that the return value has the designated type, but at + runtime we intentionally don't check anything (we want this + to be as fast as possible). + +.. decorator:: overload + + The ``@overload`` decorator allows describing functions and methods + that support multiple different combinations of argument types. A series + of ``@overload``-decorated definitions must be followed by exactly one + non-``@overload``-decorated definition (for the same function/method). + The ``@overload``-decorated definitions are for the benefit of the + type checker only, since they will be overwritten by the + non-``@overload``-decorated definition, while the latter is used at + runtime but should be ignored by a type checker. At runtime, calling + a ``@overload``-decorated function directly will raise + :exc:`NotImplementedError`. An example of overload that gives a more + precise type than can be expressed using a union or a type variable:: + + @overload + def process(response: None) -> None: + ... + @overload + def process(response: int) -> Tuple[int, str]: + ... + @overload + def process(response: bytes) -> str: + ... + def process(response): + <actual implementation> + + See :pep:`484` for details and comparison with other typing semantics. + +.. decorator:: final + + A decorator to indicate to type checkers that the decorated method + cannot be overridden, and the decorated class cannot be subclassed. + For example:: + + class Base: + @final + def done(self) -> None: + ... + class Sub(Base): + def done(self) -> None: # Error reported by type checker + ... + + @final + class Leaf: + ... + class Other(Leaf): # Error reported by type checker + ... + + There is no runtime checking of these properties. See :pep:`591` for + more details. + + .. versionadded:: 3.8 + +.. decorator:: no_type_check + + Decorator to indicate that annotations are not type hints. + + This works as class or function :term:`decorator`. With a class, it + applies recursively to all methods defined in that class (but not + to methods defined in its superclasses or subclasses). + + This mutates the function(s) in place. + +.. decorator:: no_type_check_decorator + + Decorator to give another decorator the :func:`no_type_check` effect. + + This wraps the decorator with something that wraps the decorated + function in :func:`no_type_check`. + +.. decorator:: type_check_only + + Decorator to mark a class or function to be unavailable at runtime. + + This decorator is itself not available at runtime. It is mainly + intended to mark classes that are defined in type stub files if + an implementation returns an instance of a private class:: + + @type_check_only + class Response: # private or not available at runtime + code: int + def get_header(self, name: str) -> str: ... + + def fetch_response() -> Response: ... + + Note that returning instances of private classes is not recommended. + It is usually preferable to make such classes public. + +Introspection helpers +--------------------- + +.. function:: get_type_hints(obj, globalns=None, localns=None, include_extras=False) + + Return a dictionary containing type hints for a function, method, module + or class object. + + This is often the same as ``obj.__annotations__``. In addition, + forward references encoded as string literals are handled by evaluating + them in ``globals`` and ``locals`` namespaces. If necessary, + ``Optional[t]`` is added for function and method annotations if a default + value equal to ``None`` is set. For a class ``C``, return + a dictionary constructed by merging all the ``__annotations__`` along + ``C.__mro__`` in reverse order. + + The function recursively replaces all ``Annotated[T, ...]`` with ``T``, + unless ``include_extras`` is set to ``True`` (see :class:`Annotated` for + more information). For example:: + + class Student(NamedTuple): + name: Annotated[str, 'some marker'] + + get_type_hints(Student) == {'name': str} + get_type_hints(Student, include_extras=False) == {'name': str} + get_type_hints(Student, include_extras=True) == { + 'name': Annotated[str, 'some marker'] + } + + .. versionchanged:: 3.9 + Added ``include_extras`` parameter as part of :pep:`593`. + +.. function:: get_args(tp) +.. function:: get_origin(tp) + + Provide basic introspection for generic types and special typing forms. + + For a typing object of the form ``X[Y, Z, ...]`` these functions return + ``X`` and ``(Y, Z, ...)``. If ``X`` is a generic alias for a builtin or + :mod:`collections` class, it gets normalized to the original class. + For unsupported objects return ``None`` and ``()`` correspondingly. + Examples:: + + assert get_origin(Dict[str, int]) is dict + assert get_args(Dict[int, str]) == (int, str) + + assert get_origin(Union[int, str]) is Union + assert get_args(Union[int, str]) == (int, str) + + .. versionadded:: 3.8 + +.. class:: ForwardRef + + A class used for internal typing representation of string forward references. + For example, ``List["SomeClass"]`` is implicitly transformed into + ``List[ForwardRef("SomeClass")]``. This class should not be instantiated by + a user, but may be used by introspection tools. + +Constant +-------- + +.. data:: TYPE_CHECKING + + A special constant that is assumed to be ``True`` by 3rd party static + type checkers. It is ``False`` at runtime. Usage:: + + if TYPE_CHECKING: + import expensive_mod + + def fun(arg: 'expensive_mod.SomeType') -> None: + local_var: expensive_mod.AnotherType = other_fun() + + The first type annotation must be enclosed in quotes, making it a + "forward reference", to hide the ``expensive_mod`` reference from the + interpreter runtime. Type annotations for local variables are not + evaluated, so the second annotation does not need to be enclosed in quotes. + + .. note:: + + If ``from __future__ import annotations`` is used in Python 3.7 or later, + annotations are not evaluated at function definition time. + Instead, the are stored as strings in ``__annotations__``, + This makes it unnecessary to use quotes around the annotation. + (see :pep:`563`). + + .. versionadded:: 3.5.2 - .. versionadded:: 3.9 diff --git a/Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst b/Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst new file mode 100644 index 0000000000000..b0ca4327ad61a --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2020-07-21-15-23-30.bpo-40979.pLA8rO.rst @@ -0,0 +1 @@ +Refactored typing.rst, arranging more than 70 classes, functions, and decorators into new sub-sections. \ No newline at end of file

1 0

random module: Convert a "while 1" to "while True (GH-21700)
by Raymond Hettinger Aug. 2, 2020

Aug. 2, 2020

https://github.com/python/cpython/commit/6a613f90bf13038255bca028feeab6fad8… commit: 6a613f90bf13038255bca028feeab6fad816edfd branch: master author: Raymond Hettinger <rhettinger(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2020-08-02T12:03:32-07:00 summary: random module: Convert a "while 1" to "while True (GH-21700) files: M Lib/random.py diff --git a/Lib/random.py b/Lib/random.py index 37f71110403ad..3ea369b81b3e5 100644 --- a/Lib/random.py +++ b/Lib/random.py @@ -682,7 +682,7 @@ def gammavariate(self, alpha, beta): bbb = alpha - LOG4 ccc = alpha + ainv - while 1: + while True: u1 = random() if not 1e-7 < u1 < 0.9999999: continue

1 0

bpo-41421: Algebraic simplification for random.paretovariate() (GH-21695)
by Raymond Hettinger Aug. 1, 2020

Aug. 1, 2020

https://github.com/python/cpython/commit/5c3270939c09e4c8e80fd26449b718a998… commit: 5c3270939c09e4c8e80fd26449b718a998701912 branch: master author: Raymond Hettinger <rhettinger(a)users.noreply.github.com> committer: GitHub <noreply(a)github.com> date: 2020-08-01T01:18:26-07:00 summary: bpo-41421: Algebraic simplification for random.paretovariate() (GH-21695) files: A Misc/NEWS.d/next/Library/2020-08-01-00-51-15.bpo-41421.dHKRVB.rst M Lib/random.py diff --git a/Lib/random.py b/Lib/random.py index a6454f520df0a..37f71110403ad 100644 --- a/Lib/random.py +++ b/Lib/random.py @@ -749,7 +749,7 @@ def paretovariate(self, alpha): # Jain, pg. 495 u = 1.0 - self.random() - return 1.0 / u ** (1.0 / alpha) + return u ** (-1.0 / alpha) def weibullvariate(self, alpha, beta): """Weibull distribution. diff --git a/Misc/NEWS.d/next/Library/2020-08-01-00-51-15.bpo-41421.dHKRVB.rst b/Misc/NEWS.d/next/Library/2020-08-01-00-51-15.bpo-41421.dHKRVB.rst new file mode 100644 index 0000000000000..cf291c60d8ad5 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2020-08-01-00-51-15.bpo-41421.dHKRVB.rst @@ -0,0 +1,3 @@ +Make an algebraic simplification to random.paretovariate(). It now is +slightly less subject to round-off error and is slightly faster. Inputs that +used to cause ZeroDivisionError now cause an OverflowError instead.

1 0