[Python-checkins] python/nondist/peps pep-0000.txt,1.219,1.220 pep-0288.txt,1.2,1.3
rhettinger@projects.sourceforge.net
rhettinger@projects.sourceforge.net
Mon, 18 Nov 2002 02:40:46 -0800
Update of /cvsroot/python/python/nondist/peps
In directory sc8-pr-cvs1:/tmp/cvs-serv31218
Modified Files:
pep-0000.txt pep-0288.txt
Log Message:
Revise and undefer PEP 288: Generator Attributes and Exceptions
Eliminated the original idea of passing arguments through gen.next().
Eliminated the alternative idea of using gen.submit(data).
Replaced with a developed version of generator attributes.
Left the discussion of exceptions essentially unchanged.
Index: pep-0000.txt
===================================================================
RCS file: /cvsroot/python/python/nondist/peps/pep-0000.txt,v
retrieving revision 1.219
retrieving revision 1.220
diff -C2 -d -r1.219 -r1.220
*** pep-0000.txt 8 Nov 2002 03:02:26 -0000 1.219
--- pep-0000.txt 18 Nov 2002 10:40:44 -0000 1.220
***************
*** 95,98 ****
--- 95,99 ----
S 286 Enhanced Argument Tuples von Loewis
I 287 reStructuredText Docstring Format Goodger
+ S 288 Generators Attributes and Exceptions Hettinger
I 291 Backward Compatibility for Standard Library Norwitz
S 292 Simpler String Substitutions Warsaw
***************
*** 164,168 ****
SR 270 uniq method for list objects Petrone
SR 271 Prefixing sys.path by command line option Giacometti
- SD 288 Generators Attributes and Exceptions Hettinger
SR 289 Generator Comprehensions Hettinger
SR 295 Interpretation of multiline string constants Koltsov
--- 165,168 ----
***************
*** 282,286 ****
S 286 Enhanced Argument Tuples von Loewis
I 287 reStructuredText Docstring Format Goodger
! SD 288 Generators Attributes and Exceptions Hettinger
SR 289 Generator Comprehensions Hettinger
I 290 Code Migration and Modernization Hettinger
--- 282,286 ----
S 286 Enhanced Argument Tuples von Loewis
I 287 reStructuredText Docstring Format Goodger
! S 288 Generators Attributes and Exceptions Hettinger
SR 289 Generator Comprehensions Hettinger
I 290 Code Migration and Modernization Hettinger
Index: pep-0288.txt
===================================================================
RCS file: /cvsroot/python/python/nondist/peps/pep-0288.txt,v
retrieving revision 1.2
retrieving revision 1.3
diff -C2 -d -r1.2 -r1.3
*** pep-0288.txt 5 Apr 2002 19:42:56 -0000 1.2
--- pep-0288.txt 18 Nov 2002 10:40:44 -0000 1.3
***************
*** 4,8 ****
Last-Modified: $Date$
Author: python@rcn.com (Raymond D. Hettinger)
! Status: Deferred
Type: Standards Track
Created: 21-Mar-2002
--- 4,8 ----
Last-Modified: $Date$
Author: python@rcn.com (Raymond D. Hettinger)
! Status: Draft
Type: Standards Track
Created: 21-Mar-2002
***************
*** 22,26 ****
generator tools in PEP 279 [7]. They were split-off to this
separate PEP to allow the ideas more time to mature and for
! alternatives to be considered.
--- 22,27 ----
generator tools in PEP 279 [7]. They were split-off to this
separate PEP to allow the ideas more time to mature and for
! alternatives to be considered. Subsequently, the argument
! passing idea gave way to Detlef Lannert's idea of using attributes.
***************
*** 36,48 ****
Generators, as proposed in PEP 255 [1], were introduced as a means for
making it easier to create iterators, especially ones with complex
! internal execution or variable states.
!
! The next step in the evolution of generators is to extend the
! syntax of the 'yield' keyword to enable generator parameter
! passing. The resulting increase in power simplifies the creation
! of consumer streams which have a complex execution state and/or
! variable state.
! A better alternative being considered is to allow generators to
accept attribute assignments. This allows data to be passed in a
standard Python fashion.
--- 37,43 ----
Generators, as proposed in PEP 255 [1], were introduced as a means for
making it easier to create iterators, especially ones with complex
! internal states.
! The next step in the evolution of generators is to allow generators to
accept attribute assignments. This allows data to be passed in a
standard Python fashion.
***************
*** 53,57 ****
Also, generator exception passing helps mitigate the try/finally
prohibition for generators.
!
These suggestions are designed to take advantage of the existing
implementation and require little additional effort to
--- 48,52 ----
Also, generator exception passing helps mitigate the try/finally
prohibition for generators.
!
These suggestions are designed to take advantage of the existing
implementation and require little additional effort to
***************
*** 60,118 ****
! Reference Implementation
!
! There is not currently a CPython implementation; however, a
! simulation module written in pure Python is available on
! SourceForge [5]. The simulation is meant to allow direct
! experimentation with the proposal.
!
! There is also a module [6] with working source code for all of the
! examples used in this PEP. It serves as a test suite for the
! simulator and it documents how each of the feature works in
! practice.
!
! The authors and implementers of PEP 255 [1] were contacted to
! provide their assessment of whether the enhancement was going to
! be straight-forward to implement and require only minor
! modification of the existing generator code. Neil felt the
! assertion was correct. Ka-Ping thought so also. GvR said he
! could believe that it was true. Tim did not have an opportunity
! to give an assessment.
!
! Specification for Generator Parameter Passing
! 1. Allow 'yield' to assign a value as in:
def mygen():
! while 1:
! x = yield None
! print x
!
! 2. Let the .next() method take a value to pass to the generator as in:
g = mygen()
! g.next() # runs the generator until the first 'yield'
! g.next(1) # '1' is bound to 'x' in mygen(), then printed
! g.next(2) # '2' is bound to 'x' in mygen(), then printed
The control flow of 'yield' and 'next' is unchanged by this
! proposal. The only change is that a value can be sent into the
generator. By analogy, consider the quality improvement from
GOSUB (which had no argument passing mechanism) to modern
procedure calls (which can pass in arguments and return values).
-
Most of the underlying machinery is already in place, only the
! communication needs to be added by modifying the parse syntax to
! accept the new 'x = yield expr' syntax and by allowing the .next()
! method to accept an optional argument.
Yield is more than just a simple iterator creator. It does
something else truly wonderful -- it suspends execution and saves
state. It is good for a lot more than writing iterators. This
! proposal further expands its capability by making it easier to
share data with the generator.
! The .next(arg) mechanism is especially useful for:
1. Sending data to any generator
2. Writing lazy consumers with complex execution states
--- 55,92 ----
! Specification for Generator Attributes
! Essentially, the proposal is to emulate attribute writing for classes.
! The only wrinkle is that generators lack a way to refer to instances of
! themselves. So, generators need an automatic instance variable, __self__.
! Here is a minimal example:
def mygen():
! while True:
! print __self__.data
! yield None
g = mygen()
! g.data = 1
! g.next() # prints 1
! g.data = 2
! g.next() # prints 2
The control flow of 'yield' and 'next' is unchanged by this
! proposal. The only change is that data can be sent into the
generator. By analogy, consider the quality improvement from
GOSUB (which had no argument passing mechanism) to modern
procedure calls (which can pass in arguments and return values).
Most of the underlying machinery is already in place, only the
! __self__ variable needs to be added.
Yield is more than just a simple iterator creator. It does
something else truly wonderful -- it suspends execution and saves
state. It is good for a lot more than writing iterators. This
! proposal further taps its capabilities by making it easier to
share data with the generator.
! The attribute mechanism is especially useful for:
1. Sending data to any generator
2. Writing lazy consumers with complex execution states
***************
*** 125,139 ****
queues. A class-based approach competes well when there are no
complex execution states or variable states. However, when the
! complexity increases, generators with parameter passing are much
simpler because they automatically save state (unlike classes
which must explicitly save the variable and execution state in
instance variables).
- Note A: This proposal changes 'yield' from a statement to an
- expression with binding and precedence similar to lambda.
-
Examples
!
Example of a Complex Consumer
--- 99,110 ----
queues. A class-based approach competes well when there are no
complex execution states or variable states. However, when the
! complexity increases, generators with writable attributes are much
simpler because they automatically save state (unlike classes
which must explicitly save the variable and execution state in
instance variables).
Examples
!
Example of a Complex Consumer
***************
*** 148,165 ****
def filelike(packagename, appendOrOverwrite):
! cum = []
if appendOrOverwrite == 'w+':
! cum.extend(packages[packagename])
try:
! while 1:
! dat = yield None
! cum.append(dat)
except FlushStream:
! packages[packagename] = cum
! ostream = filelike('mydest','w') # Analogous to file.open(name,flag)
! ostream.next() # Advance to the first yield
! ostream.next(firstdat) # Analogous to file.write(dat)
! ostream.next(seconddat)
ostream.throw(FlushStream) # Throw is proposed below
--- 119,135 ----
def filelike(packagename, appendOrOverwrite):
! data = []
if appendOrOverwrite == 'w+':
! data.extend(packages[packagename])
try:
! while True:
! data.append(__self__.dat)
! yield None
except FlushStream:
! packages[packagename] = data
! ostream = filelike('mydest','w')
! ostream.dat = firstdat; ostream.next()
! ostream.dat = firstdat; ostream.next()
ostream.throw(FlushStream) # Throw is proposed below
***************
*** 182,283 ****
'Analogy to Linux style pipes: source | upper | sink'
sink = sinkgen()
- sink.next()
for word in source():
! sink.next(word.upper())
!
!
! Comments
!
! Comments from GvR: We discussed this at length when we were hashing
! out generators and coroutines, and found that there's always a
! problem with this: the argument to the first next() call has
! to be thrown away, because it doesn't correspond to a yield
! statement. This looks ugly (note that the example code has a
! dummy call to next() to get the generator going). But there
! may be useful examples that can only be programmed (elegantly)
! with this feature, so I'm reserving judgment. I can believe
! that it's easy to implement.
!
! Comments from Ka-Ping Yee: I also think there is a lot of power to be
! gained from generator argument passing.
- Comments from Neil Schemenauer: I like the idea of being able to pass
- values back into a generator. I originally pitched this idea
- to Guido but in the end we decided against it (at least for
- the initial implementation). There was a few issues to work
- out but I can't seem to remember what they were. My feeling
- is that we need to wait until the Python community has more
- experience with generators before adding this feature. Maybe
- for 2.4 but not for 2.3. In the mean time you can work around
- this limitation by making your generator a method. Values can
- be passed back by mutating the instance.
! Comments for Magnus Lie Hetland: I like the generator parameter
! passing mechanism. Although I see no need to defer it,
! deferral seems to be the most likely scenario, and in the
! meantime I guess the functionality can be emulated either by
! implementing the generator as a method, or by passing a
! parameter with the exception passing mechanism.
!
! Author response: Okay, consider this proposal deferred until version 2.4
! so the idea can fully mature. I am currently teasing out two
! alternatives which may eliminate the issue with the initial
! next() call not having a corresponding yield.
! Alternative 1: Submit
!
! Instead of next(arg), use a separate method, submit(arg).
! Submit would behave just like next() except that on the first
! call, it will call next() twice. The word 'submit' has the
! further advantage of being explicit in its intent. It also
! allows checking for the proper number of arguments (next
! always has zero and submit always has one). Using this
! alternative, the call to the consumer stream looks like this:
! ostream = filelike('mydest','w')
! ostream.submit(firstdat) # No call to next is needed
! ostream.submit(seconddat)
! ostream.throw(FlushStream) # Throw is proposed below
! Alternative 2: Generator Attributes
!
! Instead of generator parameter passing, enable writable
! generator attributes: g.data=firstdat; g.next(). The code on
! the receiving end is written knowing that the attribute is set
! from the very beginning. This solves the problem because the
! first next call does not need to be associated with a yield
! statement.
! This solution uses a standard Python tool, object attributes,
! in a standard way. It is also explicit in its intention and
! provides some error checking (the receiving code raises an
! AttributeError if the expected field has not be set before the
! call).
!
! The one unclean part of this approach is that the generator
! needs some way to reference itself (something parallel to the
! use of the function name in a recursive function or to the use
! of 'self' in a method). The only way I can think of is to
! introduce a new system variable, __self__, in any function
! that employs a yield statement. Using this alternative, the
! code for the consumer stream looks like this:
! def filelike(packagename, appendOrOverwrite):
! cum = []
! if appendOrOverwrite == 'w+':
! cum.extend(packages[packagename])
! try:
! while 1:
! cum.append(__self__.dat)
! yield None
! except FlushStream:
! packages[packagename] = cum
! ostream = filelike('mydest','w')
! ostream.dat = firstdat; ostream.next()
! ostream.dat = firstdat; ostream.next()
! ostream.throw(FlushStream) # Throw is proposed in PEP 279
--- 152,190 ----
'Analogy to Linux style pipes: source | upper | sink'
sink = sinkgen()
for word in source():
! sink.data = word.upper()
! sink.next()
! Initialization Mechanism
+ If the attribute passing idea is accepted, Detlef Lannert further
+ proposed that generator instances have attributes initialized to
+ values in the generator's func_dict. This makes it easy to set
+ default values. For example:
! def mygen():
! while True:
! print __self__.data
! yield None
! mygen.data = 0
! g = mygen() # g initialized with .data set to 0
! g.next() # prints 0
! g.data = 1
! g.next() # prints 1
! Rejected Alternative
! One idea for passing data into a generator was to pass an argument
! through next() and make a assignment using the yield keyword:
! datain = yield dataout
! . . .
! dataout = gen.next(datain)
! The intractable problem is that the argument to the first next() call
! has to be thrown away, because it doesn't correspond to a yield keyword.
***************
*** 290,294 ****
log = []
try:
! while 1:
log.append( time.time() - start )
yield log[-1]
--- 197,201 ----
log = []
try:
! while True:
log.append( time.time() - start )
yield log[-1]
***************
*** 303,308 ****
There is no existing work-around for triggering an exception
! inside a generator. This is a true deficiency. It is the only
! case in Python where active code cannot be excepted to or through.
Generator exception passing also helps address an intrinsic
--- 210,215 ----
There is no existing work-around for triggering an exception
! inside a generator. It is the only case in Python where active
! code cannot be excepted to or through.
Generator exception passing also helps address an intrinsic
***************
*** 375,385 ****
THEN the need to raise exceptions will come up frequently.
- I'm no judge of what is truly Pythonic, but am still
- astonished that there can exist blocks of code that can't be
- excepted to or through, that the try/finally combination is
- blocked, and that the only work-around is to rewrite as a
- class and move the exception code out of the function or
- method being excepted.
-
References
--- 282,285 ----
***************
*** 398,411 ****
http://www.pythonware.com/products/pil/
- [5] A pure Python simulation of every feature in this PEP is at:
- http://sourceforge.net/tracker/download.php?group_id=5470&atid=305470&file_id=17348&aid=513752
-
- [6] The full, working source code for each of the examples in this PEP
- along with other examples and tests is at:
- http://sourceforge.net/tracker/download.php?group_id=5470&atid=305470&file_id=17412&aid=513756
-
- [7] PEP 279 Enhanced Generators
- http://www.python.org/peps/pep-0279.html
-
Copyright
--- 298,301 ----
***************
*** 414,418 ****
!
Local Variables:
mode: indented-text
--- 304,308 ----
!
Local Variables:
mode: indented-text