[Python-ideas] PEP 484 (Type Hints) -- first draft round

Andrew Barnert abarnert at yahoo.com
Fri Jan 16 21:14:38 CET 2015


One quick clarification question:

Can strings-as-forward-declarations be used inside non-string type annotations--e.g.. Set['Employee'] instead of 'Set[Employee]'? If so, do they have identical meaning?

I can imagine, say, a refactoring tool that didn't understand the full Python+MyPy syntax being able to do things with the former that it couldn't with the latter. And I think it might look more natural for simple recursive types. But those are just vague feelings, so don't take this as a suggestion.

Sent from a random iPhone

On Jan 16, 2015, at 9:17, Guido van Rossum <guido at python.org> wrote:

> After a long editing process we've got PEP 484 (Type Hints) ready for your review. This is by no means final, and several areas are either open (to be resolved in a later draft) or postponed (to a different PEP altogether). But there's enough meat that I think we can start having the discussion. Please also see PEP 483 (The Theory of Type Hint; copied and reformatted from the original Quip document that I posted just before last Christmas) and PEP 482 (Literature Overview for Type Hints, by Łukasz). Those are informational PEPs though; the actual spec is focused in PEP 484 (the only one on the Standards Track).
> 
> As I said earlier, I hope to have a rough consensus before PyCon and working code (just the typing.py module, really) committed to CPython before the last 3.5 alpha.
> 
> Here is the raw text of PEP 484. Fire away!!
> 
> PEP: 484
> Title: Type Hints
> Version: $Revision$
> Last-Modified: $Date$
> Author: Guido van Rossum <guido at python.org>, Jukka Lehtosalo <jukka.lehtosalo at iki.fi>, Łukasz Langa <lukasz at langa.pl>
> Discussions-To: Python-Dev <python-dev at python.org>
> Status: Draft
> Type: Standards Track
> Content-Type: text/x-rst
> Created: 29-Sep-2014
> Post-History: 16-Jan-2015
> Resolution:
> 
> 
> Abstract
> ========
> 
> This PEP introduces a standard syntax for type hints using annotations
> on function definitions.
> 
> The proposal is strongly inspired by mypy [mypy]_.
> 
> The theory behind type hints and gradual typing is explained in PEP 483.
> 
> 
> Rationale and Goals
> ===================
> 
> PEP 3107 added support for arbitrary annotations on parts of a function
> definition.  Although no meaning was assigned to annotations then, there
> has always been an implicit goal to use them for type hinting, which is
> listed as the first possible use case in said PEP.
> 
> This PEP aims to provide a standard syntax for type annotations, opening
> up Python code to easier static analysis and refactoring, potential
> runtime type checking, and performance optimizations utilizing type
> information.
> 
> 
> Type Definition Syntax
> ======================
> 
> The syntax leverages PEP 3107-style annotations with a number of
> extensions described in sections below.  In its basic form, type hinting
> is used by filling function annotations with classes::
> 
>   def greeting(name: str) -> str:
>       return 'Hello ' + name
> 
> This denotes that the expected type of the ``name`` argument is ``str``.
> Analogically, the expected return type is ``str``.  Subclasses of
> a specified argument type are also accepted as valid types for that
> argument.
> 
> Abstract base classes, types available in the ``types`` module, and
> user-defined classes may be used as type hints as well.  Annotations
> must be valid expressions that evaluate without raising exceptions at
> the time the function is defined.  In addition, the needs of static
> analysis require that annotations must be simple enough to be
> interpreted by static analysis tools.  (This is an intentionally
> somewhat vague requirement.)
> 
> .. FIXME: Define rigorously what is/isn't supported.
> 
> When used as an annotation, the expression ``None`` is considered
> equivalent to ``NoneType`` (i.e., ``type(None)`` for type hinting
> purposes.
> 
> Type aliases are also valid type hints::
> 
>   integer = int
> 
>   def retry(url: str, retry_count: integer): ...
> 
> New names that are added to support features described in following
> sections are available in the ``typing`` package.
> 
> 
> Callbacks
> ---------
> 
> Frameworks expecting callback functions of specific signatures might be
> type hinted using ``Callable[[Arg1Type, Arg2Type], ReturnType]``.
> Examples::
> 
>   from typing import Any, AnyArgs, Callable
> 
>   def feeder(get_next_item: Callable[[], Item]): ...
> 
>   def async_query(on_success: Callable[[int], None], on_error: Callable[[int, Exception], None]): ...
> 
>   def partial(func: Callable[AnyArgs, Any], *args): ...
> 
> Since using callbacks with keyword arguments is not perceived as
> a common use case, there is currently no support for specifying keyword
> arguments with ``Callable``.
> 
> 
> Generics
> --------
> 
> Since type information about objects kept in containers cannot be
> statically inferred in a generic way, abstract base classes have been
> extended to support subscription to denote expected types for container
> elements.  Example::
> 
>   from typing import Mapping, Set
> 
>   def notify_by_email(employees: Set[Employee], overrides: Mapping[str, str]): ...
> 
> Generics can be parametrized by using a new factory available in
> ``typing`` called ``TypeVar``.  Example::
> 
>   from typing import Sequence, TypeVar
> 
>   T = TypeVar('T')      # Declare type variable
> 
>   def first(l: Sequence[T]) -> T:   # Generic function
>       return l[0]
> 
> In this case the contract is that the returning value is consistent with
> the elements held by the collection.
> 
> ``TypeVar`` supports constraining parametric types to classes with any of
> the specified bases.  Example::
> 
>   from typing import Iterable
> 
>   X = TypeVar('X')
>   Y = TypeVar('Y', Iterable[X])
> 
>   def filter(rule: Callable[[X], bool], input: Y) -> Y:
>       ...
> 
> .. FIXME: Add an example with multiple bases defined.
> 
> In the example above we specify that ``Y`` can be any subclass of
> Iterable with elements of type ``X``, as long as the return type of
> ``filter()`` will be the same as the type of the ``input``
> argument.
> 
> .. FIXME: Explain more about how this works.
> 
> 
> Forward references
> ------------------
> 
> When a type hint contains names that have not been defined yet, that
> definition may be expressed as a string, to be resolved later.  For
> example, instead of writing::
> 
>   def notify_by_email(employees: Set[Employee]): ...
> 
> one might write::
> 
>   def notify_by_email(employees: 'Set[Employee]'): ...
> 
> .. FIXME: Rigorously define this.  Defend it, or find an alternative.
> 
> 
> Union types
> -----------
> 
> Since accepting a small, limited set of expected types for a single
> argument is common, there is a new special factory called ``Union``.
> Example::
> 
>   from typing import Union
> 
>   def handle_employees(e: Union[Employee, Sequence[Employee]]):
>       if isinstance(e, Employee):
>           e = [e]
>       ...
> 
> A type factored by ``Union[T1, T2, ...]`` responds ``True`` to
> ``issubclass`` checks for ``T1`` and any of its subclasses, ``T2`` and
> any of its subclasses, and so on.
> 
> One common case of union types are *optional* types.  By default,
> ``None`` is an invalid value for any type, unless a default value of
> ``None`` has been provided in the function definition.  Examples::
> 
>   def handle_employee(e: Union[Employee, None]): ...
> 
> As a shorthand for ``Union[T1, None]`` you can write ``Optional[T1]``;
> for example, the above is equivalent to::
> 
>   from typing import Optional
> 
>   def handle_employee(e: Optional[Employee]): ...
> 
> An optional type is also automatically assumed when the default value is
> ``None``, for example::
> 
>   def handle_employee(e: Employee = None): ...
> 
> This is equivalent to::
> 
>   def handle_employee(e: Optional[Employee] = None): ...
> 
> .. FIXME: Is this really a good idea?
> 
> A special kind of union type is ``Any``, a class that responds
> ``True`` to ``issubclass`` of any class.  This lets the user
> explicitly state that there are no constraints on the type of a
> specific argument or return value.
> 
> 
> Platform-specific type checking
> -------------------------------
> 
> In some cases the typing information will depend on the platform that
> the program is being executed on.  To enable specifying those
> differences, simple conditionals can be used::
> 
>   from typing import PY2, WINDOWS
> 
>   if PY2:
>       text = unicode
>   else:
>       text = str
> 
>   def f() -> text: ...
> 
>   if WINDOWS:
>       loop = ProactorEventLoop
>   else:
>       loop = UnixSelectorEventLoop
> 
> Arbitrary literals defined in the form of ``NAME = True`` will also be
> accepted by the type checker to differentiate type resolution::
> 
>   DEBUG = False
>   ...
>   if DEBUG:
>       class Tracer:
>           <verbose implementation>
>   else:
>       class Tracer:
>           <dummy implementation>
> 
> For the purposes of type hinting, the type checker assumes ``__debug__``
> is set to ``True``, in other words the ``-O`` command-line option is not
> used while type checking.
> 
> 
> Compatibility with other uses of function annotations
> -----------------------------------------------------
> 
> A number of existing or potential use cases for function annotations
> exist, which are incompatible with type hinting.  These may confuse a
> static type checker.  However, since type hinting annotations have no
> run time behavior (other than evaluation of the annotation expression
> and storing annotations in the ``__annotations__`` attribute of the
> function object), this does not make the program incorrect -- it just
> makes it issue warnings when a static analyzer is used.
> 
> To mark portions of the program that should not be covered by type
> hinting, use the following:
> 
> * a ``@no_type_checks`` decorator on classes and functions
> 
> * a ``# type: ignore`` comment on arbitrary lines
> 
> .. FIXME: should we have a module-wide comment as well?
> 
> 
> Type Hints on Local and Global Variables
> ========================================
> 
> No first-class syntax support for explicitly marking variables as being
> of a specific type is added by this PEP.  To help with type inference in
> complex cases, a comment of the following format may be used::
> 
>   x = []   # type: List[Employee]
> 
> In the case where type information for a local variable is needed before
> if was declared, an ``Undefined`` placeholder might be used::
> 
>   from typing import Undefined
> 
>   x = Undefined   # type: List[Employee]
>   y = Undefined(int)
> 
> If type hinting proves useful in general, a syntax for typing variables
> may be provided in a future Python version.
> 
> 
> Explicit raised exceptions
> ==========================
> 
> No support for listing explicitly raised exceptions is being defined by
> this PEP.  Currently the only known use case for this feature is
> documentational, in which case the recommendation is to put this
> information in a docstring.
> 
> 
> The ``typing`` package
> ======================
> 
> To open the usage of static type checking to Python 3.5 as well as older
> versions, a uniform namespace is required.  For this purpose, a new
> package in the standard library is introduced called ``typing``.  It
> holds a set of classes representing builtin types with generics, namely:
> 
> * Dict, used as ``Dict[key_type, value_type]``
> 
> * List, used as ``List[element_type]``
> 
> * Set, used as ``Set[element_type]``. See remark for ``AbstractSet``
>   below.
> 
> * FrozenSet, used as ``FrozenSet[element_type]``
> 
> * Tuple, used as ``Tuple[index0_type, index1_type, ...]``.
>   Arbitrary-length tuples might be expressed using ellipsis, in which
>   case the following arguments are considered the same type as the last
>   defined type on the tuple.
> 
> It also introduces factories and helper members needed to express
> generics and union types:
> 
> * Any, used as ``def get(key: str) -> Any: ...``
> 
> * Union, used as ``Union[Type1, Type2, Type3]``
> 
> * TypeVar, used as ``X = TypeVar('X', Type1, Type2, Type3)`` or simply
>   ``Y = TypeVar('Y')``
> 
> * Undefined, used as ``local_variable = Undefined # type: List[int]`` or
>   ``local_variable = Undefined(List[int])`` (the latter being slower
>   during runtime)
> 
> * Callable, used as ``Callable[[Arg1Type, Arg2Type], ReturnType]``
> 
> * AnyArgs, used as ``Callable[AnyArgs, ReturnType]``
> 
> * AnyStr, equivalent to ``TypeVar('AnyStr', str, bytes)``
> 
> All abstract base classes available in ``collections.abc`` are
> importable from the ``typing`` package, with added generics support:
> 
> * ByteString
> 
> * Callable
> 
> * Container
> 
> * Hashable
> 
> * ItemsView
> 
> * Iterable
> 
> * Iterator
> 
> * KeysView
> 
> * Mapping
> 
> * MappingView
> 
> * MutableMapping
> 
> * MutableSequence
> 
> * MutableSet
> 
> * Sequence
> 
> * Set as ``AbstractSet``. This name change was required because ``Set``
>   in the ``typing`` module means ``set()`` with generics.
> 
> * Sized
> 
> * ValuesView
> 
> * Mapping
> 
> The library includes literals for platform-specific type hinting:
> 
> * PY2
> 
> * PY3, equivalent to ``not PY2``
> 
> * WINDOWS
> 
> * UNIXOID, equivalent to ``not WINDOWS``
> 
> The following types are available in the ``typing.io`` module:
> 
> * IO
> 
> * BinaryIO
> 
> * TextIO
> 
> The following types are provided by the ``typing.re`` module:
> 
> * Match and Pattern, types of ``re.match()`` and ``re.compile()``
>   results
> 
> As a convenience measure, types from ``typing.io`` and ``typing.re`` are
> also available in ``typing`` (quoting Guido, "There's a reason those
> modules have two-letter names.").
> 
> 
> The place of the ``typing`` module in the standard library
> ----------------------------------------------------------
> 
> .. FIXME: complete this section
> 
> 
> Usage Patterns
> ==============
> 
> The main use case of type hinting is static analysis using an external
> tool without executing the analyzed program.  Existing tools used for
> that purpose like ``pyflakes`` [pyflakes]_ or ``pylint`` [pylint]_
> might be extended to support type checking.  New tools, like mypy's
> ``mypy -S`` mode, can be adopted specifically for this purpose.
> 
> Type checking based on type hints is understood as a best-effort
> mechanism.  In other words, whenever types are not annotated and cannot
> be inferred, the type checker considers such code valid.  Type errors
> are only reported in case of explicit or inferred conflict.  Moreover,
> as a mechanism that is not tied to execution of the code, it does not
> affect runtime behaviour.  In other words, even in the case of a typing
> error, the program will continue running.
> 
> The implementation of a type checker, whether linting source files or
> enforcing type information during runtime, is out of scope for this PEP.
> 
> .. FIXME: Describe stub modules.
> 
> .. FIXME: Describe run-time behavior of generic types.
> 
> 
> Existing Approaches
> ===================
> 
> PEP 482 lists existing approaches in Python and other languages.
> 
> 
> Is type hinting Pythonic?
> =========================
> 
> Type annotations provide important documentation for how a unit of code
> should be used.  Programmers should therefore provide type hints on
> public APIs, namely argument and return types on functions and methods
> considered public.  However, because types of local and global variables
> can be often inferred, they are rarely necessary.
> 
> The kind of information that type hints hold has always been possible to
> achieve by means of docstrings.  In fact, a number of formalized
> mini-languages for describing accepted arguments have evolved.  Moving
> this information to the function declaration makes it more visible and
> easier to access both at runtime and by static analysis.  Adding to that
> the notion that “explicit is better than implicit”, type hints are
> indeed *Pythonic*.
> 
> 
> Acknowledgements
> ================
> 
> This document could not be completed without valuable input,
> encouragement and advice from Jim Baker, Jeremy Siek, Michael Matson
> Vitousek, Andrey Vlasovskikh, and Radomir Dopieralski.
> 
> Influences include existing languages, libraries and frameworks
> mentioned in PEP 482.  Many thanks to their creators, in alphabetical
> order: Stefan Behnel, William Edwards, Greg Ewing, Larry Hastings,
> Anders Hejlsberg, Alok Menghrajani, Travis E. Oliphant, Joe Pamer,
> Raoul-Gabriel Urma, and Julien Verlaguet.
> 
> 
> References
> ==========
> 
> .. [mypy]
>    http://mypy-lang.org
> 
> .. [pyflakes]
>    https://github.com/pyflakes/pyflakes/
> 
> .. [pylint]
>    http://www.pylint.org
> 
> 
> Copyright
> =========
> 
> This document has been placed in the public domain.
> 
> 
> 
> ..
>    Local Variables:
>    mode: indented-text
>    indent-tabs-mode: nil
>    sentence-end-double-space: t
>    fill-column: 70
>    coding: utf-8
>    End:
> 
> 
> -- 
> --Guido van Rossum (python.org/~guido)
> _______________________________________________
> Python-ideas mailing list
> Python-ideas at python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-ideas/attachments/20150116/ef78852d/attachment-0001.html>


More information about the Python-ideas mailing list