[Python-checkins] python/nondist/peps pep-0234.txt,1.18,1.19

gvanrossum@users.sourceforge.net gvanrossum@users.sourceforge.net
Thu, 18 Jul 2002 13:38:31 -0700


Update of /cvsroot/python/python/nondist/peps
In directory usw-pr-cvs1:/tmp/cvs-serv2163

Modified Files:
	pep-0234.txt 
Log Message:
Add a number of clarifications and updates.


Index: pep-0234.txt
===================================================================
RCS file: /cvsroot/python/python/nondist/peps/pep-0234.txt,v
retrieving revision 1.18
retrieving revision 1.19
diff -C2 -d -r1.18 -r1.19
*** pep-0234.txt	18 Jul 2002 20:00:21 -0000	1.18
--- pep-0234.txt	18 Jul 2002 20:38:28 -0000	1.19
***************
*** 67,79 ****
        be propagated normally.
  
!     In addition to the tp_iternext slot, every iterator object must
!     also implement a next() method, callable without arguments.  This
!     should have the same semantics as the tp_iternext slot function,
!     except that the only way to signal the end of the iteration is to
!     raise StopIteration.  The iterator object should not care whether
!     its tp_iternext slot function is called or its next() method, and
!     the caller may mix calls arbitrarily.  (The next() method is for
!     the benefit of Python code using iterators directly; the
!     tp_iternext slot is added to make 'for' loops more efficient.)
  
      To ensure binary backwards compatibility, a new flag
--- 67,80 ----
        be propagated normally.
  
!     Iterators implemented in C should *not* implement a next() method
!     with similar semantics as the tp_iternext slot!  When the type's
!     dictionary is initialized (by PyType_Ready()), the presence of a
!     tp_iternext slot causes a method next() wrapping that slot to be
!     added to the type's tp_dict.  (Exception: if the type doesn't use
!     PyObject_GenericGetAttr() to access instance attributes, the
!     next() method in the type's tp_dict may not be seen.)  (Due to a
!     misunderstanding in the original text of this PEP, in Python 2.2,
!     all iterator types implemented a next() method that was overridden
!     by the wrapper; this has been fixed in Python 2.3.)
  
      To ensure binary backwards compatibility, a new flag
***************
*** 84,88 ****
      set and has a non-NULL tp_iternext slot.  There is no such macro
      for the tp_iter slot (since the only place where this slot is
!     referenced should be PyObject_GetIter()).
  
      (Note: the tp_iter slot can be present on any object; the
--- 85,90 ----
      set and has a non-NULL tp_iternext slot.  There is no such macro
      for the tp_iter slot (since the only place where this slot is
!     referenced should be PyObject_GetIter(), and this can check for
!     the Py_TPFLAGS_HAVE_ITER flag directly).
  
      (Note: the tp_iter slot can be present on any object; the
***************
*** 108,111 ****
--- 110,121 ----
      use an iterator (as opposed to a sequence) in a for loop.
  
+     Iterator implementations (in C or in Python) should guarantee that
+     once the iterator has signalled its exhaustion, subsequent calls
+     to tp_iternext or to the next() method will continue to do so.  It
+     is not specified whether an iterator should enter the exhausted
+     state when an exception (other than StopIteration) is raised.
+     Note that Python cannot guarantee that user-defined or 3rd party
+     iterators implement this requirement correctly.
+ 
  
  Python API Specification
***************
*** 164,171 ****
  Dictionary Iterators
  
-     The following two proposals are somewhat controversial.  They are
-     also independent from the main iterator implementation.  However,
-     they are both very useful.
- 
      - Dictionaries implement a sq_contains slot that implements the
        same test as the has_key() method.  This means that we can write
--- 174,177 ----
***************
*** 205,210 ****
        dict.iterkeys()".
  
!     If this proposal is accepted, it makes sense to recommend that
!     other mappings, if they support iterators at all, should also
      iterate over the keys.  However, this should not be taken as an
      absolute rule; specific applications may have different
--- 211,215 ----
        dict.iterkeys()".
  
!     Other mappings, if they support iterators at all, should also
      iterate over the keys.  However, this should not be taken as an
      absolute rule; specific applications may have different
***************
*** 214,222 ****
  File Iterators
  
!     The following proposal is not controversial, but should be
!     considered a separate step after introducing the iterator
!     framework described above.  It is useful because it provides us
!     with a good answer to the complaint that the common idiom to
!     iterate over the lines of a file is ugly and slow.
  
      - Files implement a tp_iter slot that is equivalent to
--- 219,225 ----
  File Iterators
  
!     The following proposal is useful because it provides us with a
!     good answer to the complaint that the common idiom to iterate over
!     the lines of a file is ugly and slow.
  
      - Files implement a tp_iter slot that is equivalent to
***************
*** 246,249 ****
--- 249,279 ----
      the open file object really represents a pipe or a stream socket.
  
+     Because the file iterator uses an internal buffer, mixing this
+     with other file operations (e.g. file.readline()) doesn't work
+     right.  Also, the following code:
+ 
+       for line in file:
+           if line == "\n":
+               break
+       for line in file:
+           print line,
+ 
+     doesn't work as you might expect, because the iterator created by
+     the second for-loop doesn't take the buffer read-ahead by the
+     first for-loop into account.  A correct way to write this is:
+ 
+       it = iter(file)
+       for line in it:
+           if line == "\n":
+               break
+       for line in it:
+           print line,
+ 
+     (The rationale for these restrictions are that "for line in file"
+     ought to become the recommended, standard way to iterate over the
+     lines of a file, and this should be as fast as can be.  The
+     iterator version is considerable faster than calling readline(),
+     due to the internal buffer in the iterator.)
+ 
  
  Rationale
***************
*** 294,300 ****
        than clarity.
  
      - Some folks have requested the ability to restart an iterator.
        This should be dealt with by calling iter() on a sequence
!       repeatedly, not by the iterator protocol itself.
  
      - It has been questioned whether an exception to signal the end of
--- 324,336 ----
        than clarity.
  
+       (In retrospect, it might have been better to go for __next__()
+       and have a new built-in, next(it), which calls it.__next__().
+       But alas, it's too late; this has been deployed in Python 2.2
+       since December 2001.)
+ 
      - Some folks have requested the ability to restart an iterator.
        This should be dealt with by calling iter() on a sequence
!       repeatedly, not by the iterator protocol itself.  (See also
!       requested extensions below.)
  
      - It has been questioned whether an exception to signal the end of
***************
*** 362,365 ****
--- 398,406 ----
        continues to raise StopIteration.
  
+       Note: this was in fact not implemented in Python 2.2; there are
+       many cases where an iterator's next() method can raise
+       StopIteration on one call but not on the next.  This has been
+       remedied in Python 2.3.
+ 
      - It has been proposed that a file object should be its own
        iterator, with a next() method returning the next line.  This
***************
*** 369,373 ****
        StopIteration" feature proposed in the previous bullet.
  
!       Resolution: this has been implemented.
  
      - Some folks have requested extensions of the iterator protocol,
--- 410,415 ----
        StopIteration" feature proposed in the previous bullet.
  
!       Resolution: tentatively rejected (though there are still people
!       arguing for this).
  
      - Some folks have requested extensions of the iterator protocol,
***************
*** 387,391 ****
        Resolution: rejected.
  
!     - There is still discussion about whether
  
            for x in dict: ...
--- 429,433 ----
        Resolution: rejected.
  
!     - There has been a long discussion about whether
  
            for x in dict: ...