[Python-checkins] peps: New section with contextlib example (Isaac Schwabacher).

guido.van.rossum python-checkins at python.org
Thu Nov 27 07:07:37 CET 2014


https://hg.python.org/peps/rev/bb0f11810471
changeset:   5623:bb0f11810471
user:        Guido van Rossum <guido at python.org>
date:        Wed Nov 26 22:07:29 2014 -0800
summary:
  New section with contextlib example (Isaac Schwabacher).
Other additions by Chris, including renumbered references.

files:
  pep-0479.txt |  105 ++++++++++++++++++++++++++++++--------
  1 files changed, 82 insertions(+), 23 deletions(-)


diff --git a/pep-0479.txt b/pep-0479.txt
--- a/pep-0479.txt
+++ b/pep-0479.txt
@@ -37,12 +37,39 @@
 generator to terminate silently.  (When another exception is raised, a
 traceback is printed pinpointing the cause of the problem.)
 
-The proposal also clears up the confusion about how to terminate a
-generator: the proper way is ``return``, not ``raise StopIteration``.
+This is particularly pernicious in combination with the ``yield from``
+construct of PEP 380 [1]_, as it breaks the abstraction that a
+subgenerator may be factored out of a generator.  That PEP notes this
+limitation, but notes that "use cases for these [are] rare to non-
+existent".  Unfortunately while intentional use is rare, it is easy to
+stumble on these cases by accident::
 
-Finally, the proposal reduces the difference between list
+    @contextlib.contextmanager
+    def transaction():
+        begin()
+        try:
+            yield from do_it()
+        except:
+            rollback()
+            raise
+        else:
+            commit()
+    
+    def do_it():
+        initial_preparations()
+        yield
+        finishing_touches()
+
+Here factoring out ``do_it`` into a subgenerator has introduced a
+subtle bug: if the wrapped block raises ``StopIteration``, under the
+current behavior ``do_it`` will fail but report success by returning
+normally, causing the failed transaction to be committed!  Similarly
+problematic behavior occurs when an ``asyncio`` coroutine raises
+``StopIteration``, causing it to terminate silently.
+
+Additionally, the proposal reduces the difference between list
 comprehensions and generator expressions, preventing surprises such as
-the one that started this discussion [1]_.  Henceforth, the following
+the one that started this discussion [2]_.  Henceforth, the following
 statements will produce the same result if either produces a result at
 all::
 
@@ -56,6 +83,10 @@
 will raise an exception at this point (albeit ``RuntimeError`` in the
 first case and ``StopIteration`` in the second).
 
+Finally, the proposal also clears up the confusion about how to
+terminate a generator: the proper way is ``return``, not
+``raise StopIteration``.
+
 
 Background information
 ======================
@@ -77,7 +108,7 @@
 If a ``StopIteration`` is about to bubble out of a generator frame, it
 is replaced with ``RuntimeError``, which causes the ``next()`` call
 (which invoked the generator) to fail, passing that exception out.
-From then on it's just like any old exception. [3]_
+From then on it's just like any old exception. [4]_
 
 This affects the third outcome listed above, without altering any
 other effects.  Furthermore, it only affects this outcome when the
@@ -119,10 +150,10 @@
 
 This change will affect existing code that depends on
 ``StopIteration`` bubbling up.  The pure Python reference
-implementation of ``groupby`` [2]_ currently has comments "Exit on
+implementation of ``groupby`` [3]_ currently has comments "Exit on
 ``StopIteration``" where it is expected that the exception will
 propagate and then be handled.  This will be unusual, but not unknown,
-and such constructs will fail.  Other examples abound, e.g. [5]_, [6]_.
+and such constructs will fail.  Other examples abound, e.g. [6]_, [7]_.
 
 (Nick Coghlan comments: """If you wanted to factor out a helper
 function that terminated the generator you'd have to do "return
@@ -235,6 +266,31 @@
 to find the end marker in the inner loop, which will now raise
 ``RuntimeError``).
 
+This effect of ``StopIteration`` has been used to cut a generator
+expression short, creating a form of ``takewhile``::
+
+    def stop():
+        raise StopIteration
+    print(list(x for x in range(10) if x < 5 or stop()))
+    # prints [0, 1, 2, 3, 4]
+
+Under the current proposal, this form of non-local flow control is
+not supported, and would have to be rewritten in statement form::
+
+    def gen():
+        for x in range(10):
+            if x >= 5: return
+            yield x
+    print(list(gen()))
+    # prints [0, 1, 2, 3, 4]
+
+While this is a small loss of functionality, it is functionality that
+often comes at the cost of readability, and just as ``lambda`` has
+restrictions compared to ``def``, so does a generator expression have
+restrictions compared to a generator function. In many cases, the
+transformation to full generator function will be trivially easy, and
+may improve structural clarity.
+
 
 Explanation of generators, iterators, and StopIteration
 =======================================================
@@ -330,7 +386,7 @@
 return statement.
 
 The inspiration for this alternative proposal was Nick's observation
-[7]_ that if an ``asyncio`` coroutine [8]_ accidentally raises
+[8]_ that if an ``asyncio`` coroutine [9]_ accidentally raises
 ``StopIteration``, it currently terminates silently, which may present
 a hard-to-debug mystery to the developer.  The main proposal turns
 such accidents into clearly distinguishable ``RuntimeError`` exceptions,
@@ -367,7 +423,7 @@
 Converting the exception inside next()
 --------------------------------------
 
-Mark Shannon suggested [11]_ that the problem could be solved in
+Mark Shannon suggested [12]_ that the problem could be solved in
 ``next()`` rather than at the boundary of generator functions.  By
 having ``next()`` catch ``StopIteration`` and raise instead
 ``ValueError``, all unexpected ``StopIteration`` bubbling would be
@@ -384,11 +440,11 @@
 =========
 
 Unofficial and apocryphal statistics suggest that this is seldom, if
-ever, a problem. [4]_  Code does exist which relies on the current
-behaviour (e.g. [2]_, [5]_, [6]_), and there is the concern that this
+ever, a problem. [5]_  Code does exist which relies on the current
+behaviour (e.g. [3]_, [6]_, [7]_), and there is the concern that this
 would be unnecessary code churn to achieve little or no gain.
 
-Steven D'Aprano started an informal survey on comp.lang.python [9]_;
+Steven D'Aprano started an informal survey on comp.lang.python [10]_;
 at the time of writing only two responses have been received: one was
 in favor of changing list comprehensions to match generator
 expressions (!), the other was in favor of this PEP's main proposal.
@@ -411,37 +467,40 @@
 References
 ==========
 
-.. [1] Initial mailing list comment
+.. [1] PEP 380 - Syntax for Delegating to a Subgenerator
+   (https://www.python.org/dev/peps/pep-0380)
+
+.. [2] Initial mailing list comment
    (https://mail.python.org/pipermail/python-ideas/2014-November/029906.html)
 
-.. [2] Pure Python implementation of groupby
+.. [3] Pure Python implementation of groupby
    (https://docs.python.org/3/library/itertools.html#itertools.groupby)
 
-.. [3] Proposal by GvR
+.. [4] Proposal by GvR
    (https://mail.python.org/pipermail/python-ideas/2014-November/029953.html)
 
-.. [4] Response by Steven D'Aprano
+.. [5] Response by Steven D'Aprano
    (https://mail.python.org/pipermail/python-ideas/2014-November/029994.html)
 
-.. [5] Split a sequence or generator using a predicate
+.. [6] Split a sequence or generator using a predicate
    (http://code.activestate.com/recipes/578416-split-a-sequence-or-generator-using-a-predicate/)
 
-.. [6] wrap unbounded generator to restrict its output
+.. [7] wrap unbounded generator to restrict its output
    (http://code.activestate.com/recipes/66427-wrap-unbounded-generator-to-restrict-its-output/)
 
-.. [7] Post from Nick Coghlan mentioning asyncio
+.. [8] Post from Nick Coghlan mentioning asyncio
    (https://mail.python.org/pipermail/python-ideas/2014-November/029961.html)
 
-.. [8] Coroutines in asyncio
+.. [9] Coroutines in asyncio
    (https://docs.python.org/3/library/asyncio-task.html#coroutines)
 
-.. [9] Thread on comp.lang.python started by Steven D'Aprano
+.. [10] Thread on comp.lang.python started by Steven D'Aprano
    (https://mail.python.org/pipermail/python-list/2014-November/680757.html)
 
-.. [10] Tracker issue with Proof-of-Concept patch
+.. [11] Tracker issue with Proof-of-Concept patch
    (http://bugs.python.org/issue22906)
 
-.. [11] Post from Mark Shannon with alternate proposal
+.. [12] Post from Mark Shannon with alternate proposal
    (https://mail.python.org/pipermail/python-dev/2014-November/137129.html)
 
 Copyright

-- 
Repository URL: https://hg.python.org/peps


More information about the Python-checkins mailing list