[Python-checkins] python/nondist/peps pep-0288.txt,1.3,1.4

rhettinger at users.sourceforge.net rhettinger at users.sourceforge.net
Sun Jan 2 22:41:56 CET 2005

Update of /cvsroot/python/python/nondist/peps
In directory sc8-pr-cvs1.sourceforge.net:/tmp/cvs-serv7108

Modified Files:
Log Message:
Update for Py2.5.

Index: pep-0288.txt
RCS file: /cvsroot/python/python/nondist/peps/pep-0288.txt,v
retrieving revision 1.3
retrieving revision 1.4
diff -u -d -r1.3 -r1.4
--- pep-0288.txt	18 Nov 2002 10:40:44 -0000	1.3
+++ pep-0288.txt	2 Jan 2005 21:41:54 -0000	1.4
@@ -6,186 +6,77 @@
 Status: Draft
 Type: Standards Track
 Created: 21-Mar-2002
-Python-Version: 2.4
+Python-Version: 2.5
-    This PEP introduces ideas for enhancing the generators introduced
-    in Python version 2.2 [1].  The goal is to increase the
-    convenience, utility, and power of generators by providing a
-    mechanism for passing data into a generator and for triggering
-    exceptions inside a generator.
+    This PEP proposes to enhance generators by providing mechanisms for
+    raising exceptions and sharing data with running generators.
-    These mechanisms were first proposed along with two other
-    generator tools in PEP 279 [7].  They were split-off to this
-    separate PEP to allow the ideas more time to mature and for
-    alternatives to be considered.  Subsequently, the argument
-    passing idea gave way to Detlef Lannert's idea of using attributes.
-    Python 2.2 introduced the concept of an iterable interface as
-    proposed in PEP 234 [2].  The iter() factory function was provided
-    as common calling convention and deep changes were made to use
-    iterators as a unifying theme throughout Python.  The unification
-    came in the form of establishing a common iterable interface for
-    mappings, sequences, and file objects.
-    Generators, as proposed in PEP 255 [1], were introduced as a means for
-    making it easier to create iterators, especially ones with complex
-    internal states.
+    Currently, only class based iterators can provide attributes and
+    exception handling.  However, class based iterators are harder to
+    write, less compact, less readable, and slower.  A better solution
+    is to enable these capabilities for generators.
-    The next step in the evolution of generators is to allow generators to
-    accept attribute assignments.  This allows data to be passed in a
-    standard Python fashion.
+    Enabling attribute assignments allows data to be passed to and from
+    running generators.  The approach of sharing data using attributes
+    pervades Python.  Other approaches exist but are somewhat hackish
+    in comparison.
-    A related evolutionary step is to add a generator method to enable
+    Another evolutionary step is to add a generator method to allow
     exceptions to be passed to a generator.  Currently, there is no
     clean method for triggering exceptions from outside the generator.
     Also, generator exception passing helps mitigate the try/finally
-    prohibition for generators.
+    prohibition for generators.  The need is especially acute for
+    generators needing to flush buffers or close resources upon termination.
+    The two proposals are backwards compatible and require no new
+    keywords.  They are being recommended for Python version 2.5.
-    These suggestions are designed to take advantage of the existing
-    implementation and require little additional effort to
-    incorporate.  They are backwards compatible and require no new
-    keywords.  They are being recommended for Python version 2.4.
 Specification for Generator Attributes
     Essentially, the proposal is to emulate attribute writing for classes.
     The only wrinkle is that generators lack a way to refer to instances of
-    themselves.  So, generators need an automatic instance variable, __self__.
-    Here is a minimal example:
-        def mygen():
-            while True:
-                print __self__.data
-                yield None
-        g = mygen()
-        g.data = 1
-        g.next()                # prints 1
-        g.data = 2
-        g.next()                # prints 2
-    The control flow of 'yield' and 'next' is unchanged by this
-    proposal.  The only change is that data can be sent into the
-    generator.  By analogy, consider the quality improvement from
-    GOSUB (which had no argument passing mechanism) to modern
-    procedure calls (which can pass in arguments and return values).
-    Most of the underlying machinery is already in place, only the
-    __self__ variable needs to be added.
-    Yield is more than just a simple iterator creator.  It does
-    something else truly wonderful -- it suspends execution and saves
-    state.  It is good for a lot more than writing iterators.  This
-    proposal further taps its capabilities by making it easier to
-    share data with the generator.
-    The attribute mechanism is especially useful for:
-        1. Sending data to any generator
-        2. Writing lazy consumers with complex execution states
-        3. Writing co-routines (as demonstrated in Dr. Mertz's articles [3])
-    The proposal is a clear improvement over the existing alternative
-    of passing data via global variables.  It is also much simpler,
-    more readable and easier to debug than an approach involving the
-    threading module with its attendant mutexes, semaphores, and data
-    queues.  A class-based approach competes well when there are no
-    complex execution states or variable states.  However, when the
-    complexity increases, generators with writable attributes are much
-    simpler because they automatically save state (unlike classes
-    which must explicitly save the variable and execution state in
-    instance variables).
-    Example of a Complex Consumer
-    The encoder for arithmetic compression sends a series of
-    fractional values to a complex, lazy consumer.  That consumer
-    makes computations based on previous inputs and only writes out
-    when certain conditions have been met.  After the last fraction is
-    received, it has a procedure for flushing any unwritten data.
-    Example of a Consumer Stream
-        def filelike(packagename, appendOrOverwrite):
-            data = []
-            if appendOrOverwrite == 'w+':
-                data.extend(packages[packagename])
-            try:
-                while True:
-                    data.append(__self__.dat)
-                    yield None
-            except FlushStream:
-                packages[packagename] = data
-        ostream = filelike('mydest','w')
-        ostream.dat = firstdat; ostream.next()
-        ostream.dat = firstdat; ostream.next()
-        ostream.throw(FlushStream)         # Throw is proposed below
-    Example of a Complex Consumer
-    Loop over the picture files in a directory, shrink them one at a
-    time to thumbnail size using PIL [4], and send them to a lazy
-    consumer.  That consumer is responsible for creating a large blank
-    image, accepting thumbnails one at a time and placing them in a 5
-    by 3 grid format onto the blank image.  Whenever the grid is full,
-    it writes-out the large image as an index print.  A FlushStream
-    exception indicates that no more thumbnails are available and that
-    the partial index print should be written out if there are one or
-    more thumbnails on it.
-    Example of a Producer and Consumer Used Together in a Pipe-like Fashion
-        'Analogy to Linux style pipes:  source | upper | sink'
-        sink = sinkgen()
-        for word in source():
-            sink.data = word.upper()
-            sink.next()
-Initialization Mechanism
-    If the attribute passing idea is accepted, Detlef Lannert further
-    proposed that generator instances have attributes initialized to
-    values in the generator's func_dict.  This makes it easy to set
-    default values.  For example:
-        def mygen():
-            while True:
-                print __self__.data
-                yield None
-        mygen.data = 0
+    themselves.  So, the proposal is to provide a function for discovering
+    the reference.  For example:
-        g = mygen()             # g initialized with .data set to 0
-        g.next()                # prints 0
-        g.data = 1
-        g.next()                # prints 1
+        def mygen(filename):
+            self = mygen.get_instance()
+            myfile = open(filename)
+            for line in myfile:
+                if len(line) < 10:
+                    continue
+                self.pos = myfile.tell()
+                yield line.upper()
+        g = mygen('sample.txt')
+        line1 = g.next()
+        print 'Position', g.pos
-Rejected Alternative
+    Uses for generator attributes include:
-    One idea for passing data into a generator was to pass an argument
-    through next() and make a assignment using the yield keyword:
+        1. Providing generator clients with extra information (as shown
+           above).
+        2. Externally setting control flags governing generator operation
+           (possibly telling a generator when to step in or step over
+           data groups).
+        3. Writing lazy consumers with complex execution states
+           (an arithmetic encoder output stream for example).
+        4. Writing co-routines (as demonstrated in Dr. Mertz's articles [1]).
-        datain = yield dataout
-          . . .
-        dataout = gen.next(datain)
+    The control flow of 'yield' and 'next' is unchanged by this
+    proposal.  The only change is that data can passed to and from the
+    generator.  Most of the underlying machinery is already in place,
+    only the access function needs to be added.
-    The intractable problem is that the argument to the first next() call
-    has to be thrown away, because it doesn't correspond to a yield keyword.
 Specification for Generator Exception Passing:
@@ -197,7 +88,7 @@
             log = []
                 while True:
-                    log.append( time.time() - start )
+                    log.append(time.time() - start)
                     yield log[-1]
             except WriteLog:
@@ -214,9 +105,7 @@
     Generator exception passing also helps address an intrinsic
     limitation on generators, the prohibition against their using
-    try/finally to trigger clean-up code [1].  Without .throw(), the
-    current work-around forces the resolution or clean-up code to be
-    moved outside the generator.
+    try/finally to trigger clean-up code [2].
     Note A: The name of the throw method was selected for several
     reasons.  Raise is a keyword and so cannot be used as a method
@@ -227,10 +116,10 @@
     already associated with exceptions in other languages.
     Alternative method names were considered: resolve(), signal(),
-    genraise(), raiseinto(), and flush().  None of these seem to fit
-    as well as throw().
+    genraise(), raiseinto(), and flush().  None of these fit as well
+    as throw().
-    Note B: The throw syntax should exactly match raise's syntax:
+    Note B: The full throw() syntax should exactly match raise's syntax:
         throw([expression, [expression, [expression]]])
@@ -243,59 +132,19 @@
         raise                           g.throw()
-    Comments from GvR:  I'm not convinced that the cleanup problem that
-        this is trying to solve exists in practice. I've never felt
-        the need to put yield inside a try/except. I think the PEP
-        doesn't make enough of a case that this is useful.
-        This one gets a big fat -1 until there's a good motivational
-        section.
-    Comments from Ka-Ping Yee:  I agree that the exception issue needs to
-        be resolved and [that] you have suggested a fine solution.
-    Comments from Neil Schemenauer:  The exception passing idea is one I
-        hadn't thought of before and looks interesting.  If we enable
-        the passing of values back, then we should add this feature
-        too.
-    Comments for Magnus Lie Hetland:  Even though I cannot speak for the
-        ease of implementation, I vote +1 for the exception passing
-        mechanism.
-    Comments from the Community:  The response has been mostly favorable.  One
-        negative comment from GvR is shown above.  The other was from
-        Martin von Loewis who was concerned that it could be difficult
-        to implement and is withholding his support until a working
-        patch is available.  To probe Martin's comment, I checked with
-        the implementers of the original generator PEP for an opinion
-        on the ease of implementation.  They felt that implementation
-        would be straight-forward and could be grafted onto the
-        existing implementation without disturbing its internals.
-    Author response:  When the sole use of generators is to simplify writing
-        iterators for lazy producers, then the odds of needing
-        generator exception passing are slim.  If, on the other hand,
-        generators are used to write lazy consumers, create
-        coroutines, generate output streams, or simply for their
-        marvelous capability for restarting a previously frozen state,
-        THEN the need to raise exceptions will come up frequently.
-    [1] PEP 255 Simple Generators
-        http://www.python.org/peps/pep-0255.html
-    [2] PEP 234 Iterators
-        http://www.python.org/peps/pep-0234.html
-    [3] Dr. David Mertz's draft column for Charming Python.
+    [1] Dr. David Mertz's draft columns for Charming Python:
-    [4] PIL, the Python Imaging Library can be found at:
-        http://www.pythonware.com/products/pil/
+    [2] PEP 255 Simple Generators:
+        http://www.python.org/peps/pep-0255.html
+    [3] Proof-of-concept recipe:
+        http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/164044

More information about the Python-checkins mailing list