[Python-checkins] python/nondist/peps pep-0289.txt,1.4,1.5

Thu Oct 23 02:36:39 EDT 2003

Update of /cvsroot/python/python/nondist/peps
In directory sc8-pr-cvs1:/tmp/cvs-serv24085

Modified Files:
	pep-0289.txt 
Log Message:
Add lots of detail and examples to the Details section.  Free
variables are now captured at definition time.

Index: pep-0289.txt
===================================================================
RCS file: /cvsroot/python/python/nondist/peps/pep-0289.txt,v
retrieving revision 1.4
retrieving revision 1.5
diff -C2 -d -r1.4 -r1.5
*** pep-0289.txt	22 Oct 2003 22:08:06 -0000	1.4
--- pep-0289.txt	23 Oct 2003 06:36:37 -0000	1.5
***************
*** 66,70 ****

      dotproduct = sum(x*y for x,y in itertools.izip(x_vector, y_vector))
!     
  Having a syntax similar to list comprehensions also makes it easy to
  convert existing code into an generator expression when scaling up
--- 66,70 ----

      dotproduct = sum(x*y for x,y in itertools.izip(x_vector, y_vector))
! 
  Having a syntax similar to list comprehensions also makes it easy to
  convert existing code into an generator expression when scaling up
***************
*** 86,99 ****
  ===========

! 1.  The semantics of a generator expression are equivalent to creating
! an anonymous generator function and calling it.  There's still discussion
! about whether that generator function should copy the current value of
! all free variables into default arguments.

! 2. The syntax requires that a generator expression always needs to be inside
! a set of parentheses and cannot have a comma on either side.  Unfortunately,
! this is different from list comprehensions.  While [1, x for x in R] is
! illegal, [x for x in 1, 2, 3] is legal, meaning [x for x in (1,2,3)].
! With reference to the file Grammar/Grammar in CVS, two rules change:

      a) The rule::
--- 86,112 ----
  ===========

! (None of this is exact enough in the eye of a reader from Mars, but I
! hope the examples convey the intention well enough for a discussion in
! c.l.py.  The Python Reference Manual should contain a 100% exact
! semantic and syntactic specification.)

! 1. The semantics of a generator expression are equivalent to creating
! an anonymous generator function and calling it.  For example::
! 
!     g = (x**2 for x in range(10))
!     print g.next()
! 
! is equivalent to::
! 
!     def __gen():
!         for x in range(10):
!             yield x**2
!     g = __gen()
!     print g.next()
! 
! 2. The syntax requires that a generator expression always needs to be
! directly inside a set of parentheses and cannot have a comma on either
! side.  With reference to the file Grammar/Grammar in CVS, two rules
! change:

      a) The rule::
***************
*** 110,140 ****
      b)  The rule for arglist needs similar changes.

! 2. The loop variable is not exposed to the surrounding function.  This
! facilates the implementation and makes typical use cases more reliable.
! In some future version of Python, list comprehensions will also hide the
  induction variable from the surrounding code (and, in Py2.4, warnings
  will be issued for code accessing the induction variable).
-                                                                 
- 3. There is still discussion about whether variable referenced in generator
- expressions will exhibit late binding just like other Python code.  In the
- following example, the iterator runs *after* the value of y is set to one::

!     def h():
!         y = 0
!         l = [1,2]
!         def gen(S):
!             for x in S:
!                 yield x+y
!         it = gen(l)
!         y = 1
!         for v in it:
!           print v

! 4. List comprehensions will remain unchanged::

      [x for x in S]    # This is a list comprehension.
      [(x for x in S)]  # This is a list containing one generator expression.

  Reduction Functions
--- 123,212 ----
      b)  The rule for arglist needs similar changes.

+ This means that you can write::

!     sum(x**2 for x in range(10))
! 
! but you would have to write::
! 
!     reduce(operator.add, (x**2 for x in range(10)))
! 
! and also::
! 
!     g = (x**2 for i in range(10))
! 
! i.e. if a function call has a single positional argument, it can be a
! generator expression without extra parentheses, but in all other cases
! you have to parenthesize it.
! 
! 3. The loop variable (if it is a simple variable or a tuple of simple
! variables) is not exposed to the surrounding function.  This facilates
! the implementation and makes typical use cases more reliable.  In some
! future version of Python, list comprehensions will also hide the
  induction variable from the surrounding code (and, in Py2.4, warnings
  will be issued for code accessing the induction variable).

! For example::

!     x = "hello"
!     y = list(x for x in "abc")
!     print x    # prints "hello", not "c"
! 
! (Loop variables may also use constructs like x[i] or x.a; this form
! may be deprecated.)
! 
! 4. All free variable bindings are captured at the time this function
! is defined, and passed into it using default argument values.  For
! example::
! 
!     x = 0
!     g = (x for c in "abc")   # x is not the loop variable!
!     x = 1
!     print g.next()    # prints 0 (captured x), not 1 (current x)
! 
! This behavior of free variables is almost always what you want when
! the generator expression is evaluated at a later point than its
! definition.  In fact, to date, no examples have been found of code
! where it would be better to use the execution-time instead of the
! definition-time value of a free variable.
! 
! Note that free variables aren't copied, only their binding is
! captured.  They may still change if they are mutable, for example::
! 
!     x = []
!     g = (x for c in "abc")
!     x.append(1)
!     print g.next()    # prints [1], not []
! 
! 5. List comprehensions will remain unchanged.  For example::

      [x for x in S]    # This is a list comprehension.
      [(x for x in S)]  # This is a list containing one generator expression.

+ Unfortunately, there is currently a slight syntactic difference.  The
+ expression::
+ 
+     [x for x in 1, 2, 3]
+ 
+ is legal, meaning::
+ 
+     [x for x in (1, 2, 3)]
+ 
+ But generator expressions will not allow the former version::
+ 
+     (x for x in 1, 2, 3)
+ 
+ is illegal.
+ 
+ The former list comprehension syntax will become illegal in Python
+ 3.0, and should be deprecated in Python 2.4 and beyond.
+ 
+ List comprehensions also "leak" their loop variable into the
+ surrounding scope.  This will also change in Python 3.0, so that the
+ semantic definition of a list comprehension in Python 3.0 will be
+ equivalent to list(<generator expression>).  Python 2.4 and beyond
+ should issue a deprecation warning if a list comprehension's loop
+ variable has the same name as a variable used in the immediately
+ surrounding scope.
+ 

  Reduction Functions
***************
*** 153,159 ****
  * Raymond Hettinger first proposed the idea of "generator comprehensions"
    in January 2002.
!   
  * Peter Norvig resurrected the discussion in his proposal for
!   Accumulation Displays [3]_.

  * Alex Martelli provided critical measurements that proved the performance
--- 225,231 ----
  * Raymond Hettinger first proposed the idea of "generator comprehensions"
    in January 2002.
! 
  * Peter Norvig resurrected the discussion in his proposal for
!   Accumulation Displays.

  * Alex Martelli provided critical measurements that proved the performance
***************
*** 180,184 ****
  .. [3] Peter Norvig's Accumulation Display Proposal
         http:///www.norvig.com/pyacc.html
!        
  .. [4] Jeff Epler had worked up a patch demonstrating
         the previously proposed bracket and yield syntax
--- 252,256 ----
  .. [3] Peter Norvig's Accumulation Display Proposal
         http:///www.norvig.com/pyacc.html
! 
  .. [4] Jeff Epler had worked up a patch demonstrating
         the previously proposed bracket and yield syntax