[Python-checkins] peps: Remove over-specified text about finding PEP source, link instead to repo
chris.angelico
python-checkins at python.org
Mon Jan 19 23:31:18 CET 2015
https://hg.python.org/peps/rev/b93444aa3d3b
changeset: 5675:b93444aa3d3b
user: Chris Angelico <rosuav at gmail.com>
date: Tue Jan 20 09:30:07 2015 +1100
summary:
Remove over-specified text about finding PEP source, link instead to repo
files:
pep-0012.txt | 5 +-
pep-0479.txt.orig | 603 ++++++++++++++++++++++++++++++++++
pep-0483.txt.orig | 328 ++++++++++++++++++
3 files changed, 933 insertions(+), 3 deletions(-)
diff --git a/pep-0012.txt b/pep-0012.txt
--- a/pep-0012.txt
+++ b/pep-0012.txt
@@ -23,9 +23,8 @@
the text (reStructuredText) source of this PEP in order to complete
the steps below. **DO NOT USE THE HTML FILE AS YOUR TEMPLATE!**
-To get the source this (or any) PEP, look at the top of the HTML page
-and click on the date & time on the "Last-Modified" line. It is a
-link to the source text in the Python repository.
+The source for this (or any) PEP can be found in the PEPs repository,
+viewable on the web at https://hg.python.org/peps/file/tip .
If you would prefer not to use markup in your PEP, please see PEP 9,
"Sample Plaintext PEP Template" [2]_.
diff --git a/pep-0479.txt.orig b/pep-0479.txt.orig
new file mode 100644
--- /dev/null
+++ b/pep-0479.txt.orig
@@ -0,0 +1,603 @@
+PEP: 479
+Title: Change StopIteration handling inside generators
+Version: $Revision$
+Last-Modified: $Date$
+Author: Chris Angelico <rosuav at gmail.com>, Guido van Rossum <guido at python.org>
+Status: Accepted
+Type: Standards Track
+Content-Type: text/x-rst
+Created: 15-Nov-2014
+Python-Version: 3.5
+Post-History: 15-Nov-2014, 19-Nov-2014, 5-Dec-2014
+
+
+Abstract
+========
+
+This PEP proposes a change to generators: when ``StopIteration`` is
+raised inside a generator, it is replaced it with ``RuntimeError``.
+(More precisely, this happens when the exception is about to bubble
+out of the generator's stack frame.) Because the change is backwards
+incompatible, the feature is initially introduced using a
+``__future__`` statement.
+
+
+Acceptance
+==========
+
+This PEP was accepted by the BDFL on November 22. Because of the
+exceptionally short period from first draft to acceptance, the main
+objections brought up after acceptance were carefully considered and
+have been reflected in the "Alternate proposals" section below.
+However, none of the discussion changed the BDFL's mind and the PEP's
+acceptance is now final. (Suggestions for clarifying edits are still
+welcome -- unlike IETF RFCs, the text of a PEP is not cast in stone
+after its acceptance, although the core design/plan/specification
+should not change after acceptance.)
+
+
+Rationale
+=========
+
+The interaction of generators and ``StopIteration`` is currently
+somewhat surprising, and can conceal obscure bugs. An unexpected
+exception should not result in subtly altered behaviour, but should
+cause a noisy and easily-debugged traceback. Currently,
+``StopIteration`` can be absorbed by the generator construct.
+
+The main goal of the proposal is to ease debugging in the situation
+where an unguarded ``next()`` call (perhaps several stack frames deep)
+raises ``StopIteration`` and causes the iteration controlled by the
+generator to terminate silently. (When another exception is raised, a
+traceback is printed pinpointing the cause of the problem.)
+
+This is particularly pernicious in combination with the ``yield from``
+construct of PEP 380 [1]_, as it breaks the abstraction that a
+subgenerator may be factored out of a generator. That PEP notes this
+limitation, but notes that "use cases for these [are] rare to non-
+existent". Unfortunately while intentional use is rare, it is easy to
+stumble on these cases by accident::
+
+ import contextlib
+
+ @contextlib.contextmanager
+ def transaction():
+ print('begin')
+ try:
+ yield from do_it()
+ except:
+ print('rollback')
+ raise
+ else:
+ print('commit')
+
+ def do_it():
+ print('Refactored initial setup')
+ yield # Body of with-statement is executed here
+ print('Refactored finalization of successful transaction')
+
+
+ import pathlib
+
+ with transaction():
+ print('commit file {}'.format(
+ # I can never remember what the README extension is
+ next(pathlib.Path('/some/dir').glob('README*'))))
+
+Here factoring out ``do_it`` into a subgenerator has introduced a subtle
+bug: if a bug in the wrapped block allows ``StopIteration`` to escape
+(here because the README doesn't exist), under the current behavior
+``do_it`` will properly abort, but the exception will be swallowed by
+the ``yield from`` and the original context manager will commit the
+unfinished transaction! Similarly problematic behavior occurs when an
+``asyncio`` coroutine raises ``StopIteration``, causing it to terminate
+silently. In both cases, the refactoring abstraction of ``yield from``
+breaks in the presence of bugs in client code.
+
+Additionally, the proposal reduces the difference between list
+comprehensions and generator expressions, preventing surprises such as
+the one that started this discussion [2]_. Henceforth, the following
+statements will produce the same result if either produces a result at
+all::
+
+ a = list(F(x) for x in xs if P(x))
+ a = [F(x) for x in xs if P(x)]
+
+With the current state of affairs, it is possible to write a function
+``F(x)`` or a predicate ``P(x)`` that causes the first form to produce
+a (truncated) result, while the second form raises an exception
+(namely, ``StopIteration``). With the proposed change, both forms
+will raise an exception at this point (albeit ``RuntimeError`` in the
+first case and ``StopIteration`` in the second).
+
+Finally, the proposal also clears up the confusion about how to
+terminate a generator: the proper way is ``return``, not
+``raise StopIteration``.
+
+As an added bonus, the above changes bring generator functions much
+more in line with regular functions. If you wish to take a piece of
+code presented as a generator and turn it into something else, you
+can usually do this fairly simply, by replacing every ``yield`` with
+a call to ``print()`` or ``list.append()``; however, if there are any
+bare ``next()`` calls in the code, you have to be aware of them. If
+the code was originally written without relying on ``StopIteration``
+terminating the function, the transformation would be that much
+easier.
+
+
+Background information
+======================
+
+When a generator frame is (re)started as a result of a ``__next__()``
+(or ``send()`` or ``throw()``) call, one of three outcomes can occur:
+
+* A yield point is reached, and the yielded value is returned.
+* The frame is returned from; ``StopIteration`` is raised.
+* An exception is raised, which bubbles out.
+
+In the latter two cases the frame is abandoned (and the generator
+object's ``gi_frame`` attribute is set to None).
+
+
+Proposal
+========
+
+If a ``StopIteration`` is about to bubble out of a generator frame, it
+is replaced with ``RuntimeError``, which causes the ``next()`` call
+(which invoked the generator) to fail, passing that exception out.
+From then on it's just like any old exception. [4]_
+
+This affects the third outcome listed above, without altering any
+other effects. Furthermore, it only affects this outcome when the
+exception raised is ``StopIteration`` (or a subclass thereof).
+
+Note that the proposed replacement happens at the point where the
+exception is about to bubble out of the frame, i.e. after any
+``except`` or ``finally`` blocks that could affect it have been
+exited. The ``StopIteration`` raised by returning from the frame is
+not affected (the point being that ``StopIteration`` means that the
+generator terminated "normally", i.e. it did not raise an exception).
+
+A subtle issue is what will happen if the caller, having caught the
+``RuntimeError``, calls the generator object's ``__next__()`` method
+again. The answer is that from this point on it will raise
+``StopIteration`` -- the behavior is the same as when any other
+exception was raised by the generator.
+
+Another logical consequence of the proposal: if someone uses
+``g.throw(StopIteration)`` to throw a ``StopIteration`` exception into
+a generator, if the generator doesn't catch it (which it could do
+using a ``try/except`` around the ``yield``), it will be transformed
+into ``RuntimeError``.
+
+During the transition phase, the new feature must be enabled
+per-module using::
+
+ from __future__ import generator_stop
+
+Any generator function constructed under the influence of this
+directive will have the ``REPLACE_STOPITERATION`` flag set on its code
+object, and generators with the flag set will behave according to this
+proposal. Once the feature becomes standard, the flag may be dropped;
+code should not inspect generators for it.
+
+
+Consequences for existing code
+==============================
+
+This change will affect existing code that depends on
+``StopIteration`` bubbling up. The pure Python reference
+implementation of ``groupby`` [3]_ currently has comments "Exit on
+``StopIteration``" where it is expected that the exception will
+propagate and then be handled. This will be unusual, but not unknown,
+and such constructs will fail. Other examples abound, e.g. [6]_, [7]_.
+
+(Nick Coghlan comments: """If you wanted to factor out a helper
+function that terminated the generator you'd have to do "return
+yield from helper()" rather than just "helper()".""")
+
+There are also examples of generator expressions floating around that
+rely on a ``StopIteration`` raised by the expression, the target or the
+predicate (rather than by the ``__next__()`` call implied in the ``for``
+loop proper).
+
+Writing backwards and forwards compatible code
+----------------------------------------------
+
+With the exception of hacks that raise ``StopIteration`` to exit a
+generator expression, it is easy to write code that works equally well
+under older Python versions as under the new semantics.
+
+This is done by enclosing those places in the generator body where a
+``StopIteration`` is expected (e.g. bare ``next()`` calls or in some
+cases helper functions that are expected to raise ``StopIteration``)
+in a ``try/except`` construct that returns when ``StopIteration`` is
+raised. The ``try/except`` construct should appear directly in the
+generator function; doing this in a helper function that is not itself
+a generator does not work. If ``raise StopIteration`` occurs directly
+in a generator, simply replace it with ``return``.
+
+
+Examples of breakage
+--------------------
+
+Generators which explicitly raise ``StopIteration`` can generally be
+changed to simply return instead. This will be compatible with all
+existing Python versions, and will not be affected by ``__future__``.
+Here are some illustrations from the standard library.
+
+Lib/ipaddress.py::
+
+ if other == self:
+ raise StopIteration
+
+Becomes::
+
+ if other == self:
+ return
+
+In some cases, this can be combined with ``yield from`` to simplify
+the code, such as Lib/difflib.py::
+
+ if context is None:
+ while True:
+ yield next(line_pair_iterator)
+
+Becomes::
+
+ if context is None:
+ yield from line_pair_iterator
+ return
+
+(The ``return`` is necessary for a strictly-equivalent translation,
+though in this particular file, there is no further code, and the
+``return`` can be omitted.) For compatibility with pre-3.3 versions
+of Python, this could be written with an explicit ``for`` loop::
+
+ if context is None:
+ for line in line_pair_iterator:
+ yield line
+ return
+
+More complicated iteration patterns will need explicit ``try/except``
+constructs. For example, a hypothetical parser like this::
+
+ def parser(f):
+ while True:
+ data = next(f)
+ while True:
+ line = next(f)
+ if line == "- end -": break
+ data += line
+ yield data
+
+would need to be rewritten as::
+
+ def parser(f):
+ while True:
+ try:
+ data = next(f)
+ while True:
+ line = next(f)
+ if line == "- end -": break
+ data += line
+ yield data
+ except StopIteration:
+ return
+
+or possibly::
+
+ def parser(f):
+ for data in f:
+ while True:
+ line = next(f)
+ if line == "- end -": break
+ data += line
+ yield data
+
+The latter form obscures the iteration by purporting to iterate over
+the file with a ``for`` loop, but then also fetches more data from
+the same iterator during the loop body. It does, however, clearly
+differentiate between a "normal" termination (``StopIteration``
+instead of the initial line) and an "abnormal" termination (failing
+to find the end marker in the inner loop, which will now raise
+``RuntimeError``).
+
+This effect of ``StopIteration`` has been used to cut a generator
+expression short, creating a form of ``takewhile``::
+
+ def stop():
+ raise StopIteration
+ print(list(x for x in range(10) if x < 5 or stop()))
+ # prints [0, 1, 2, 3, 4]
+
+Under the current proposal, this form of non-local flow control is
+not supported, and would have to be rewritten in statement form::
+
+ def gen():
+ for x in range(10):
+ if x >= 5: return
+ yield x
+ print(list(gen()))
+ # prints [0, 1, 2, 3, 4]
+
+While this is a small loss of functionality, it is functionality that
+often comes at the cost of readability, and just as ``lambda`` has
+restrictions compared to ``def``, so does a generator expression have
+restrictions compared to a generator function. In many cases, the
+transformation to full generator function will be trivially easy, and
+may improve structural clarity.
+
+
+Explanation of generators, iterators, and StopIteration
+=======================================================
+
+Under this proposal, generators and iterators would be distinct, but
+related, concepts. Like the mixing of text and bytes in Python 2,
+the mixing of generators and iterators has resulted in certain
+perceived conveniences, but proper separation will make bugs more
+visible.
+
+An iterator is an object with a ``__next__`` method. Like many other
+special methods, it may either return a value, or raise a specific
+exception - in this case, ``StopIteration`` - to signal that it has
+no value to return. In this, it is similar to ``__getattr__`` (can
+raise ``AttributeError``), ``__getitem__`` (can raise ``KeyError``),
+and so on. A helper function for an iterator can be written to
+follow the same protocol; for example::
+
+ def helper(x, y):
+ if x > y: return 1 / (x - y)
+ raise StopIteration
+
+ def __next__(self):
+ if self.a: return helper(self.b, self.c)
+ return helper(self.d, self.e)
+
+Both forms of signalling are carried through: a returned value is
+returned, an exception bubbles up. The helper is written to match
+the protocol of the calling function.
+
+A generator function is one which contains a ``yield`` expression.
+Each time it is (re)started, it may either yield a value, or return
+(including "falling off the end"). A helper function for a generator
+can also be written, but it must also follow generator protocol::
+
+ def helper(x, y):
+ if x > y: yield 1 / (x - y)
+
+ def gen(self):
+ if self.a: return (yield from helper(self.b, self.c))
+ return (yield from helper(self.d, self.e))
+
+In both cases, any unexpected exception will bubble up. Due to the
+nature of generators and iterators, an unexpected ``StopIteration``
+inside a generator will be converted into ``RuntimeError``, but
+beyond that, all exceptions will propagate normally.
+
+
+Transition plan
+===============
+
+- Python 3.5: Enable new semantics under ``__future__`` import; silent
+ deprecation warning if ``StopIteration`` bubbles out of a generator
+ not under ``__future__`` import.
+
+- Python 3.6: Non-silent deprecation warning.
+
+- Python 3.7: Enable new semantics everywhere.
+
+
+Alternate proposals
+===================
+
+Raising something other than RuntimeError
+-----------------------------------------
+
+Rather than the generic ``RuntimeError``, it might make sense to raise
+a new exception type ``UnexpectedStopIteration``. This has the
+downside of implicitly encouraging that it be caught; the correct
+action is to catch the original ``StopIteration``, not the chained
+exception.
+
+
+Supplying a specific exception to raise on return
+-------------------------------------------------
+
+Nick Coghlan suggested a means of providing a specific
+``StopIteration`` instance to the generator; if any other instance of
+``StopIteration`` is raised, it is an error, but if that particular
+one is raised, the generator has properly completed. This subproposal
+has been withdrawn in favour of better options, but is retained for
+reference.
+
+
+Making return-triggered StopIterations obvious
+----------------------------------------------
+
+For certain situations, a simpler and fully backward-compatible
+solution may be sufficient: when a generator returns, instead of
+raising ``StopIteration``, it raises a specific subclass of
+``StopIteration`` (``GeneratorReturn``) which can then be detected.
+If it is not that subclass, it is an escaping exception rather than a
+return statement.
+
+The inspiration for this alternative proposal was Nick's observation
+[8]_ that if an ``asyncio`` coroutine [9]_ accidentally raises
+``StopIteration``, it currently terminates silently, which may present
+a hard-to-debug mystery to the developer. The main proposal turns
+such accidents into clearly distinguishable ``RuntimeError`` exceptions,
+but if that is rejected, this alternate proposal would enable
+``asyncio`` to distinguish between a ``return`` statement and an
+accidentally-raised ``StopIteration`` exception.
+
+Of the three outcomes listed above, two change:
+
+* If a yield point is reached, the value, obviously, would still be
+ returned.
+* If the frame is returned from, ``GeneratorReturn`` (rather than
+ ``StopIteration``) is raised.
+* If an instance of ``GeneratorReturn`` would be raised, instead an
+ instance of ``StopIteration`` would be raised. Any other exception
+ bubbles up normally.
+
+In the third case, the ``StopIteration`` would have the ``value`` of
+the original ``GeneratorReturn``, and would reference the original
+exception in its ``__cause__``. If uncaught, this would clearly show
+the chaining of exceptions.
+
+This alternative does *not* affect the discrepancy between generator
+expressions and list comprehensions, but allows generator-aware code
+(such as the ``contextlib`` and ``asyncio`` modules) to reliably
+differentiate between the second and third outcomes listed above.
+
+However, once code exists that depends on this distinction between
+``GeneratorReturn`` and ``StopIteration``, a generator that invokes
+another generator and relies on the latter's ``StopIteration`` to
+bubble out would still be potentially wrong, depending on the use made
+of the distinction between the two exception types.
+
+
+Converting the exception inside next()
+--------------------------------------
+
+Mark Shannon suggested [12]_ that the problem could be solved in
+``next()`` rather than at the boundary of generator functions. By
+having ``next()`` catch ``StopIteration`` and raise instead
+``ValueError``, all unexpected ``StopIteration`` bubbling would be
+prevented; however, the backward-incompatibility concerns are far
+more serious than for the current proposal, as every ``next()`` call
+now needs to be rewritten to guard against ``ValueError`` instead of
+``StopIteration`` - not to mention that there is no way to write one
+block of code which reliably works on multiple versions of Python.
+(Using a dedicated exception type, perhaps subclassing ``ValueError``,
+would help this; however, all code would still need to be rewritten.)
+
+
+Sub-proposal: decorator to explicitly request current behaviour
+---------------------------------------------------------------
+
+Nick Coghlan suggested [13]_ that the situations where the current
+behaviour is desired could be supported by means of a decorator::
+
+ from itertools import allow_implicit_stop
+
+ @allow_implicit_stop
+ def my_generator():
+ ...
+ yield next(it)
+ ...
+
+Which would be semantically equivalent to::
+
+ def my_generator():
+ try:
+ ...
+ yield next(it)
+ ...
+ except StopIteration
+ return
+
+but be faster, as it could be implemented by simply permitting the
+``StopIteration`` to bubble up directly.
+
+Single-source Python 2/3 code would also benefit in a 3.7+ world,
+since libraries like six and python-future could just define their own
+version of "allow_implicit_stop" that referred to the new builtin in
+3.5+, and was implemented as an identity function in other versions.
+
+However, due to the implementation complexities required, the ongoing
+compatibility issues created, the subtlety of the decorator's effect,
+and the fact that it would encourage the "quick-fix" solution of just
+slapping the decorator onto all generators instead of properly fixing
+the code in question, this sub-proposal has been rejected. [14]_
+
+
+Criticism
+=========
+
+Unofficial and apocryphal statistics suggest that this is seldom, if
+ever, a problem. [5]_ Code does exist which relies on the current
+behaviour (e.g. [3]_, [6]_, [7]_), and there is the concern that this
+would be unnecessary code churn to achieve little or no gain.
+
+Steven D'Aprano started an informal survey on comp.lang.python [10]_;
+at the time of writing only two responses have been received: one was
+in favor of changing list comprehensions to match generator
+expressions (!), the other was in favor of this PEP's main proposal.
+
+The existing model has been compared to the perfectly-acceptable
+issues inherent to every other case where an exception has special
+meaning. For instance, an unexpected ``KeyError`` inside a
+``__getitem__`` method will be interpreted as failure, rather than
+permitted to bubble up. However, there is a difference. Special
+methods use ``return`` to indicate normality, and ``raise`` to signal
+abnormality; generators ``yield`` to indicate data, and ``return`` to
+signal the abnormal state. This makes explicitly raising
+``StopIteration`` entirely redundant, and potentially surprising. If
+other special methods had dedicated keywords to distinguish between
+their return paths, they too could turn unexpected exceptions into
+``RuntimeError``; the fact that they cannot should not preclude
+generators from doing so.
+
+
+References
+==========
+
+.. [1] PEP 380 - Syntax for Delegating to a Subgenerator
+ (https://www.python.org/dev/peps/pep-0380)
+
+.. [2] Initial mailing list comment
+ (https://mail.python.org/pipermail/python-ideas/2014-November/029906.html)
+
+.. [3] Pure Python implementation of groupby
+ (https://docs.python.org/3/library/itertools.html#itertools.groupby)
+
+.. [4] Proposal by GvR
+ (https://mail.python.org/pipermail/python-ideas/2014-November/029953.html)
+
+.. [5] Response by Steven D'Aprano
+ (https://mail.python.org/pipermail/python-ideas/2014-November/029994.html)
+
+.. [6] Split a sequence or generator using a predicate
+ (http://code.activestate.com/recipes/578416-split-a-sequence-or-generator-using-a-predicate/)
+
+.. [7] wrap unbounded generator to restrict its output
+ (http://code.activestate.com/recipes/66427-wrap-unbounded-generator-to-restrict-its-output/)
+
+.. [8] Post from Nick Coghlan mentioning asyncio
+ (https://mail.python.org/pipermail/python-ideas/2014-November/029961.html)
+
+.. [9] Coroutines in asyncio
+ (https://docs.python.org/3/library/asyncio-task.html#coroutines)
+
+.. [10] Thread on comp.lang.python started by Steven D'Aprano
+ (https://mail.python.org/pipermail/python-list/2014-November/680757.html)
+
+.. [11] Tracker issue with Proof-of-Concept patch
+ (http://bugs.python.org/issue22906)
+
+.. [12] Post from Mark Shannon with alternate proposal
+ (https://mail.python.org/pipermail/python-dev/2014-November/137129.html)
+
+.. [13] Idea from Nick Coghlan
+ (https://mail.python.org/pipermail/python-dev/2014-November/137201.html)
+
+.. [14] Rejection by GvR
+ (https://mail.python.org/pipermail/python-dev/2014-November/137243.html)
+
+Copyright
+=========
+
+This document has been placed in the public domain.
+
+
+
+..
+ Local Variables:
+ mode: indented-text
+ indent-tabs-mode: nil
+ sentence-end-double-space: t
+ fill-column: 70
+ coding: utf-8
+ End:
diff --git a/pep-0483.txt.orig b/pep-0483.txt.orig
new file mode 100644
--- /dev/null
+++ b/pep-0483.txt.orig
@@ -0,0 +1,328 @@
+PEP: 483
+Title: The Theory of Type Hinting
+Version: $Revision$
+Last-Modified: $Date$
+Author: Guido van Rossum <guido at python.org>
+Discussions-To: Python-Ideas <python-ideas at python.org>
+Status: Draft
+Type: Informational
+Content-Type: text/x-rst
+Created: 08-Jan-2015
+Post-History:
+Resolution:
+
+The Theory of Type Hinting
+==========================
+
+Guido van Rossum, Dec 19, 2014 (with a few more recent updates)
+
+This work is licensed under a `Creative Commons
+Attribution-NonCommercial-ShareAlike 4.0 International
+License <http://creativecommons.org/licenses/by-nc-sa/4.0/>`_.
+
+
+Introduction
+------------
+
+This document lays out the theory of the new type hinting proposal for
+Python 3.5. It's not quite a full proposal or specification because
+there are many details that need to be worked out, but it lays out the
+theory without which it is hard to discuss more detailed specifications.
+We start by explaining gradual typing; then we state some conventions
+and general rules; then we define the new special types (such as Union)
+that can be used in annotations; and finally we define the approach to
+generic types. (The latter section needs more fleshing out; sorry!)
+
+
+Summary of gradual typing
+-------------------------
+
+We define a new relationship, is-consistent-with, which is similar to
+is-subclass-of, except it is not transitive when the new type **Any** is
+involved. (Neither relationship is symmetric.) Assigning x to y is OK if
+the type of x is consistent with the type of y. (Compare this to “... if
+the type of x is a subclass of the type of y,” which states one of the
+fundamentals of OO programming.) The is-consistent-with relationship is
+defined by three rules:
+
+- A type t1 is consistent with a type t2 if t1 is a subclass of t2.
+ (But not the other way around.)
+- **Any** is consistent with every type. (But **Any** is not a subclass
+ of every type.)
+- Every type is a subclass of **Any**. (Which also makes every type
+ consistent with **Any**, via rule 1.)
+
+That's all! See Jeremy Siek's blog post `What is Gradual
+Typing <http://wphomes.soic.indiana.edu/jsiek/what-is-gradual-typing/>`_
+for a longer explanation and motivation. Note that rule 3 places **Any**
+at the root of the class graph. This makes it very similar to
+**object**. The difference is that **object** is not consistent with
+most types (e.g. you can't use an object() instance where an int is
+expected). IOW both **Any** and **object** mean “any type is allowed”
+when used to annotate an argument, but only **Any** can be passed no
+matter what type is expected (in essence, **Any** shuts up complaints
+from the static checker).
+
+Here's an example showing how these rules work out in practice:
+
+Say we have an Employee class, and a subclass Manager:
+
+- class Employee: ...
+- class Manager(Employee): ...
+
+Let's say variable e is declared with type Employee:
+
+- e = Employee() # type: Employee
+
+Now it's okay to assign a Manager instance to e (rule 1):
+
+- e = Manager()
+
+It's not okay to assign an Employee instance to a variable declared with
+type Manager:
+
+- m = Manager() # type: Manager
+- m = Employee() # Fails static check
+
+However, suppose we have a variable whose type is **Any**:
+
+- a = some\_func() # type: Any
+
+Now it's okay to assign a to e (rule 2):
+
+- e = a # OK
+
+Of course it's also okay to assign e to a (rule 3), but we didn't need
+the concept of consistency for that:
+
+- a = e # OK
+
+Notational conventions
+~~~~~~~~~~~~~~~~~~~~~~
+
+- t1, t2 etc. and u1, u2 etc. are types or classes. Sometimes we write
+ ti or tj to refer to “any of t1, t2, etc.”
+- X, Y etc. are type variables (defined with Var(), see below).
+- C, D etc. are classes defined with a class statement.
+- x, y etc. are objects or instances.
+- We use the terms type and class interchangeably, and we assume
+ type(x) is x.\_\_class\_\_.
+
+General rules
+~~~~~~~~~~~~~
+
+- Instance-ness is derived from class-ness, e.g. x is an instance of
+ t1 if type(x) is a subclass of t1.
+- No types defined below (i.e. Any, Union etc.) can be instantiated.
+ (But non-abstract subclasses of Generic can be.)
+- No types defined below can be subclassed, except for Generic and
+ classes derived from it.
+- Where a type is expected, None can be substituted for type(None);
+ e.g. Union[t1, None] == Union[t1, type(None)].
+
+Types
+~~~~~
+
+- **Any**. Every class is a subclass of Any; however, to the static
+ type checker it is also consistent with every class (see above).
+- **Union[t1, t2, ...]**. Classes that are subclass of at least one of
+ t1 etc. are subclasses of this. So are unions whose components are
+ all subclasses of t1 etc. (Example: Union[int, str] is a subclass of
+ Union[int, float, str].) The order of the arguments doesn't matter.
+ (Example: Union[int, str] == Union[str, int].) If ti is itself a
+ Union the result is flattened. (Example: Union[int, Union[float,
+ str]] == Union[int, float, str].) If ti and tj have a subclass
+ relationship, the less specific type survives. (Example:
+ Union[Employee, Manager] == Union[Employee].) Union[t1] returns just
+ t1. Union[] is illegal, so is Union[()]. Corollary: Union[..., Any,
+ ...] returns Any; Union[..., object, ...] returns object; to cut a
+ tie, Union[Any, object] == Union[object, Any] == Any.
+- **Optional[t1]**. Alias for Union[t1, None], i.e. Union[t1,
+ type(None)].
+- **Tuple[t1, t2, ..., tn]**. A tuple whose items are instances of t1
+ etc.. Example: Tuple[int, float] means a tuple of two items, the
+ first is an int, the second a float; e.g., (42, 3.14). Tuple[u1, u2,
+ ..., um] is a subclass of Tuple[t1, t2, ..., tn] if they have the
+ same length (n==m) and each ui is a subclass of ti. To spell the type
+ of the empty tuple, use Tuple[()]. There is no way to define a
+ variadic tuple type. (TODO: Maybe Tuple[t1, ...] with literal
+ ellipsis?)
+- **Callable[[t1, t2, ..., tn], tr]**. A function with positional
+ argument types t1 etc., and return type tr. The argument list may be
+ empty (n==0). There is no way to indicate optional or keyword
+ arguments, nor varargs (we don't need to spell those often enough to
+ complicate the syntax — however, Reticulated Python has a useful idea
+ here). This is covariant in the return type, but contravariant in the
+ arguments. “Covariant” here means that for two callable types that
+ differ only in the return type, the subclass relationship for the
+ callable types follows that of the return types. (Example:
+ Callable[[], Manager] is a subclass of Callable[[], Employee].)
+ “Contravariant“ here means that for two callable types that differ
+ only in the type of one argument, the subclass relationship for the
+ callable types goes in the opposite direction as for the argument
+ types. (Example: Callable[[Employee], None] is a subclass of
+ Callable[[Mananger], None]. Yes, you read that right.)
+
+We might add:
+
+- **Intersection[t1, t2, ...]**. Classes that are subclass of *each* of
+ t1, etc are subclasses of this. (Compare to Union, which has *at
+ least one* instead of *each* in its definition.) The order of the
+ arguments doesn't matter. Nested intersections are flattened, e.g.
+ Intersection[int, Intersection[float, str]] == Intersection[int,
+ float, str]. An intersection of fewer types is a subclass of an
+ intersection of more types, e.g. Intersection[int, str] is a subclass
+ of Intersection[int, float, str]. An intersection of one argument is
+ just that argument, e.g. Intersection[int] is int. When argument have
+ a subclass relationship, the more specific class survives, e.g.
+ Intersection[str, Employee, Manager] is Intersection[str, Manager].
+ Intersection[] is illegal, so is Intersection[()]. Corollary: Any
+ disappears from the argument list, e.g. Intersection[int, str, Any]
+ == Intersection[int, str].Intersection[Any, object] is object. The
+ interaction between Intersection and Union is complex but should be
+ no surprise if you understand the interaction between intersections
+ and unions in set theory (note that sets of types can be infinite in
+ size, since there is no limit on the number of new subclasses).
+
+Pragmatics
+~~~~~~~~~~
+
+Some things are irrelevant to the theory but make practical use more
+convenient. (This is not a full list; I probably missed a few and some
+are still controversial or not fully specified.)
+
+Type aliases, e.g.
+
+- point = Tuple[float, float]
+- def distance(p: point) -> float: ...
+
+Forward references via strings, e.g.
+
+class C:
+
+- def compare(self, other: “C”) -> int: ...
+
+If a default of None is specified, the type is implicitly optional, e.g.
+
+- def get(key: KT, default: VT = None) -> VT: ...
+
+Don't use dynamic type expressions; use builtins and imported types
+only. No 'if'.
+
+- def display(message: str if WINDOWS else bytes): # NOT OK
+
+Type declaration in comments, e.g.
+
+- x = [] # type: Sequence[int]
+
+Type declarations using Undefined, e.g.
+
+- x = Undefined(str)
+
+Other things, e.g. casts, overloading and stub modules; best left to an
+actual PEP.
+
+Generic types
+~~~~~~~~~~~~~
+
+(TODO: Explain more. See also the `mypy docs on
+generics <http://mypy.readthedocs.org/en/latest/generics.html>`_.)
+
+**X = Var('X')**. Declares a unique type variable. The name must match
+the variable name.
+
+**Y = Var('Y', t1, t2, ...).** Ditto, constrained to t1 etc. Behaves
+ like Union[t1, t2, ...] for most purposes, but when used as a type
+variable, subclasses of t1 etc. are replaced by the most-derived base
+class among t1 etc.
+
+Example of constrained type variables:
+
+AnyStr = Var('AnyStr', str, bytes)
+
+def longest(a: AnyStr, b: AnyStr) -> AnyStr:
+
+- return a if len(a) >= len(b) else b
+
+x = longest('a', 'abc') # The inferred type for x is str
+
+y = longest('a', b'abc') # Fails static type check
+
+In this example, both arguments to longest() must have the same type
+(str or bytes), and moreover, even if the arguments are instances of a
+common str subclass, the return type is still str, not that subclass
+(see next example).
+
+For comparison, if the type variable was unconstrained, the common
+subclass would be chosen as the return type, e.g.:
+
+S = Var('S')
+
+def longest(a: S, b: S) -> S:
+
+- return a if len(a) >= b else b
+
+class MyStr(str): ...
+
+x = longest(MyStr('a'), MyStr('abc'))
+
+The inferred type of x is MyStr (whereas in the AnyStr example it would
+be str).
+
+Also for comparison, if a Union is used, the return type also has to be
+a Union:
+
+U = Union[str, bytes]
+
+def longest(a: U, b: U) -> U:
+
+- return a if len(a) >- b else b
+
+x = longest('a', 'abc')
+
+The inferred type of x is still Union[str, bytes], even though both
+arguments are str.
+
+**class C(Generic[X, Y, ...]):** ... Define a generic class C over type
+variables X etc. C itself becomes parameterizable, e.g. C[int, str, ...]
+is a specific class with substitutions X→int etc.
+
+TODO: Explain use of generic types in function signatures. E.g.
+Sequence[X], Sequence[int], Sequence[Tuple[X, Y, Z]], and mixtures.
+Think about co\*variance. No gimmicks like deriving from
+Sequence[Union[int, str]] or Sequence[Union[int, X]].
+
+**Protocol**. Similar to Generic but uses structural equivalence. (TODO:
+Explain, and think about co\*variance.)
+
+Predefined generic types and Protocols in typing.py
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+(See also the `mypy typing.py
+module <https://github.com/JukkaL/typing/blob/master/typing.py>`_.)
+
+- Everything from collections.abc (but Set renamed to AbstractSet).
+- Dict, List, Set, a few more. (FrozenSet?)
+- Pattern, Match. (Why?)
+- IO, TextIO, BinaryIO. (Why?)
+
+Another reference
+~~~~~~~~~~~~~~~~~
+
+Lest mypy gets all the attention, I should mention \ `Reticulated
+Python <https://github.com/mvitousek/reticulated>`_ by Michael Vitousek
+as an example of a slightly different approach to gradual typing for
+Python. It is described in an actual `academic
+paper <http://wphomes.soic.indiana.edu/jsiek/files/2014/03/retic-python.pdf>`_
+written by Vitousek with Jeremy Siek and Jim Baker (the latter of Jython
+fame).
+
+
+..
+ Local Variables:
+ mode: indented-text
+ indent-tabs-mode: nil
+ sentence-end-double-space: t
+ fill-column: 70
+ coding: utf-8
+ End:
--
Repository URL: https://hg.python.org/peps
More information about the Python-checkins
mailing list