[Python-checkins] CVS: python/nondist/peps pep-0279.txt,1.1,1.2
Barry Warsaw
bwarsaw@users.sourceforge.net
Fri, 01 Feb 2002 06:55:49 -0800
Update of /cvsroot/python/python/nondist/peps
In directory usw-pr-cvs1:/tmp/cvs-serv14542
Modified Files:
pep-0279.txt
Log Message:
Integrated Raymond Hettinger's latest update
Index: pep-0279.txt
===================================================================
RCS file: /cvsroot/python/python/nondist/peps/pep-0279.txt,v
retrieving revision 1.1
retrieving revision 1.2
diff -C2 -d -r1.1 -r1.2
*** pep-0279.txt 2002/02/01 05:59:14 1.1
--- pep-0279.txt 2002/02/01 14:55:46 1.2
***************
*** 15,20 ****
This PEP introduces four orthogonal (not mutually exclusive) ideas
for enhancing the generators as introduced in Python version 2.2
! [1]. The goal is increase the convenience, utility, and power of
! generators.
--- 15,20 ----
This PEP introduces four orthogonal (not mutually exclusive) ideas
for enhancing the generators as introduced in Python version 2.2
! [1]. The goal is to increase the convenience, utility, and power
! of generators.
***************
*** 116,120 ****
if not values_left[0]:
raise StopIteration
! yield func(*args)
def xzip( *collections ):
--- 116,120 ----
if not values_left[0]:
raise StopIteration
! yield fun(*args)
def xzip( *collections ):
***************
*** 136,140 ****
gen = iter(collection)
while limit is None or cnt<limit:
! yield (cnt, collection.next())
cnt += 1
--- 136,140 ----
gen = iter(collection)
while limit is None or cnt<limit:
! yield (cnt, gen.next())
cnt += 1
***************
*** 186,190 ****
really a generator is the presence of the yield keyword). On the
minus side, the brackets may falsely suggest that the whole
! expression returns a list. All of the feedback received to date
indicates that brackets do not make a false suggestion and are
in fact helpful.
--- 186,190 ----
really a generator is the presence of the yield keyword). On the
minus side, the brackets may falsely suggest that the whole
! expression returns a list. Most of the feedback received to date
indicates that brackets do not make a false suggestion and are
in fact helpful.
***************
*** 203,230 ****
g = mygen()
! g.next() # runs the generators until the first 'yield'
! g.next(1) # the '1' gets bound to 'x' in mygen()
! g.next(2) # the '2' gets bound to 'x' in mygen()
! Note A: An early question arose, when would you need this? The
! answer is that existing generators make it easy to write lazy
! producers which may have a complex execution state and/or complex
! variable state. This proposal makes it equally easy to write lazy
! consumers which may also have a complex execution or variable
! state.
! For instance, when writing an encoder for arithmetic compression,
! a series of fractional values are sent to a function which has
! periodic output and a complex state which depends on previous
! inputs. Also, that encoder requires a flush() function when no
! additional fractions are to be output. It is helpful to think of
! the following parallel with file output streams:
! ostream = file('mydest.txt','w')
! ostream.write(firstdat)
! ostream.write(seconddat)
! ostream.flush()
! With the proposed extensions, it could be written like this:
def filelike(packagename, appendOrOverwrite):
--- 203,254 ----
g = mygen()
! g.next() # runs the generator until the first 'yield'
! g.next(1) # '1' is bound to 'x' in mygen(), then printed
! g.next(2) # '2' is bound to 'x' in mygen(), then printed
! The control flow is unchanged by this proposal. The only change
! is that a value can be sent into the generator. By analogy,
! consider the quality improvement from GOSUB (which had no argument
! passing mechanism) to modern procedure calls (which pass in
! arguments and return values).
! Most of the underlying machinery is already in place, only the
! communication needs to be added by modifying the parse syntax to
! accept the new 'x = yield expr' syntax and by allowing the .next()
! method to accept an optional argument.
! Yield is more than just a simple iterator creator. It does
! something else truly wonderful -- it suspends execution and saves
! state. It is good for a lot more than writing iterators. This
! proposal further expands its capability by making it easier to
! share data with the generator.
! The .next(arg) mechanism is especially useful for:
! 1. Sending data to any generator
! 2. Writing lazy consumers with complex execution states
! 3. Writing co-routines (as demonstrated in Dr. Mertz's article [5])
!
! The proposal is a clear improvement over the existing alternative
! of passing data via global variables. It is also much simpler,
! more readable and easier to debug than an approach involving the
! threading module with its attendant mutexes, semaphores, and data
! queues. A class-based approach competes well when there are no
! complex execution states or variable states. When the complexity
! increases, generators with two-way communication are much simpler
! because they automatically save state (unlike classes which must
! explicitly save the variable and execution state in instance
! variables).
!
!
! Example of a Complex Consumer
!
! The encoder for arithmetic compression sends a series of
! fractional values to a complex, lazy consumer. That consumer
! makes computations based on previous inputs and only writes out
! when certain conditions have been met. After the last fraction is
! received, it has a procedure for flushing any unwritten data.
!
!
! Example of a Consumer Stream
def filelike(packagename, appendOrOverwrite):
***************
*** 238,345 ****
except FlushStream:
packages[packagename] = cum
! ostream = filelike('mydest','w')
! ostream.next()
! ostream.next(firstdat)
ostream.next(seconddat)
! ostream.throw( FlushStream ) # this feature discussed below
!
! Note C: Almost all of the machinery necessary to implement this
! extension is already in place. The parse syntax needs to be
! modified to accept the new x = yield None syntax and the .next()
! method needs to allow an argument.
!
! Note D: Some care must be used when writing a values to the
! generator because execution starts at the top of the generator not
! at the first yield.
!
! Consider the usual flow using .next() without an argument.
!
! g = mygen(p1) will bind p1 to a local variable and then return a
! generator to be bound to g and NOT run any code in mygen().
! y = g.next() runs the generator from the first line until it
! encounters a yield when it suspends execution and a returns
! a value to be bound to y
!
! Since the same flow applies when you are submitting values, the
! first call to .next() should have no argument since there is no
! place to put it.
!
! g = mygen(p1) will bind p1 to a local variable and then return a
! generator to be bound to g and NOT run any code in mygen()
! g.next() will START execution in mygen() from the first line. Note,
! that there is nowhere to bind any potential arguments that
! might have been supplied to next(). Execution continues
! until the first yield is encountered and control is returned
! to the caller.
! g.next(val) resumes execution at the yield and binds val to the
! left hand side of the yield assignment and continues running
! until another yield is encountered. This makes sense because
! you submit values expecting them to be processed right away.
!
!
! Q. Two-way generator parameter passing seems awfully bold. To
! my mind, one of the great things about generators is that they
! meet the (very simple) definition of an iterator. With this,
! they no longer do. I like lazy consumers -- really I do --
! but I'd rather be conservative about putting something like
! this in the language.
!
! A. If you don't use x = yield expr, then nothing changes and you
! haven't lost anything. So, it isn't really bold. It simply
! adds an option to pass in data as well as take it out. Other
! generator implementations (like the thread based generator.py)
! already have provisions for two-way parameter passing so that
! consumers are put on an equal footing with producers. Two-way
! is the norm, not the exception.
!
! Yield is not just a simple iterator creator. It does
! something else truly wonderful -- it suspends execution and
! saves state. It is good for a lot more than its original
! purpose. Dr. Mertz's article [5] shows how they can be used
! to create general purpose co-routines.
!
! Besides, 98% of the mechanism is already in place. Only the
! communication needs to be added. Remember GOSUB which neither
! took nor returned data. Routines which accepted parameters
! and returned values were a major step forward.
!
! When you first need to pass information into a generator, the
! existing alternative is clumsy. It involves setting a global
! variable, calling .next(), and assigning the local from the
! global.
!
!
! Q. Why not introduce another keyword 'accept' for lazy consumers?
!
! A. To avoid conflicts with 'yield', to avoid creating a new
! keyword, and to take advantage of the explicit clarity of the
! '=' operator.
!
!
! Q. How often does one need to write a lazy consumer or a co-routine?
!
! A. Not often. But, when you DO have to write one, this approach
! is the easiest to implement, read, and debug.
!
! It clearly beats using existing generators and passing data
! through global variables. It is much clearer and easier to
! debug than an equivalent approach using threading, mutexes,
! semaphores, and data queues. A class based approach competes
! well when there are no complex execution states or variable
! states. When the complexity increases, generators with
! two-way communication are much simpler because they
! automatically save state unlike classes which must explicitly
! store variable and execution state in instance variables.
! Q. Why does yield require an argument? Isn't yield None too wordy?
! A. It doesn't matter for the purposes of this PEP. For
! information purposes, here is the reasoning as I understand
! it. Though return allows an implicit None, some now consider
! this to be weak design. There is some spirit of "Explicit is
! better than Implicit". More importantly, in most uses of
! yield, a missing argument is more likely to be a bug than an
! intended yield None.
--- 262,283 ----
except FlushStream:
packages[packagename] = cum
! ostream = filelike('mydest','w') # Analogous to file.open(name,flag)
! ostream.next() # Advance to the first yield
! ostream.next(firstdat) # Analogous to file.write(dat)
ostream.next(seconddat)
! ostream.throw( FlushStream ) # This feature proposed below
! Example of a Complex Consumer
! Loop over the picture files in a directory, shrink them
! one-at-a-time to thumbnail size using PIL, and send them to a lazy
! consumer. That consumer is responsible for creating a large blank
! image, accepting thumbnails one-at-a-time and placing them in a
! 5x3 grid format onto the blank image. Whenever the grid is full,
! it writes-out the large image as an index print. A FlushStream
! exception indicates that no more thumbnails are available and that
! the partial index print should be written out if there are one or
! more thumbnails on it.
***************
*** 363,368 ****
inside a generator. This is a true deficiency. It is the only
case in Python where active code cannot be excepted to or through.
! Even if .next(arg) is not adopted, we should add the .throw()
! method.
Note A: The name of the throw method was selected for several
--- 301,306 ----
inside a generator. This is a true deficiency. It is the only
case in Python where active code cannot be excepted to or through.
! Even if the .next(arg) proposal is not adopted, we should add the
! .throw() method.
Note A: The name of the throw method was selected for several
***************
*** 401,405 ****
http://groups.google.com/groups?hl=en&th=215e6e5a7bfd526&rnum=2
!
http://groups.google.com/groups?hl=en&th=df8b5e7709957eb7
--- 339,343 ----
http://groups.google.com/groups?hl=en&th=215e6e5a7bfd526&rnum=2
! Discussion Draft of this PEP
http://groups.google.com/groups?hl=en&th=df8b5e7709957eb7
***************
*** 419,422 ****
fill-column: 70
End:
-
-
--- 357,358 ----