Iterators in Sather and Python (was Re: Result of I need your experience - classification and comparison of languages)

Mon May 13 12:33:23 EDT 2002

(Note: according to my understanding of Outlook Express, this should
only go to c.l.p and not c.l.r or c.l.s.   Please let me know if
otherwise.)

"Alex Martelli" <aleax at aleax.it> wrote in message
news:lOKD8.34859$CN3.1237997 at news2.tin.it...

> Sather iterators are indeed awesome and I highly recommend everybody
> to study them, see:
> http://www.icsi.berkeley.edu/~sather/Publications/toplas.html

I found this interesting and useful for better understanding iteration
both in Python and elsewhere.  But I am slightly less impressed with
the Sather alternative.  As a result of having 'loop...end' as the
only loop construct, Sather generalizes 'iterator' to 'something that
controls looping' from 'something that sequentially accesses
collection items -- which  may
control looping as a side-effect'.  Sather combine two ideas that I
think are better kept somewhat separate: loop control and sequential
presentation of implicit or explicit set/container contents.  A Sather
iterator yields nothing (not even None) for pure loop control (to go
on), something for at least implicit sequencing, or quits (equivalent
to raise StopIteration in Python).  Hence, every class inherits the
control iterators while!(), until!(), and break! even though they have
nothing in particular to do with the class and its values.  I find
this to be as much confusing as elegant.

However, Sather's generalization does suggest a *possible* proprosal
for Python: make an otherwise uncaught StopIteration consistently do
what it says within for and while loop bodies as as well as in
for-loop headers.  IE, implicitly wrap loop bodies with

try: <body>
except StopIteration: break

In particular,

item = iterator.next()

would (if not otherwise wrapped with try) act like

try: item = iterator.next()
except StopIteration: break

as it implicitly does now in for headers.  This would also allow (for
better or worse)

def until(val):
  if val: raise StopIteration

while 1:
  <do stuff>
  until(condition)
  <do more stuff>

which should mostly end requests for a do-while or until statement.

Counterargument: does not add functionality; explicit better than
implicit except in bounded context such as for loop head.

>  Traversing more than one structure at a time is easy for Sather,
not for Ruby.

Python, of course, has map and zip and proposals (accepted yet?) for
lazy, iterator/generator oriented versions.

> Python's iterators are far closer to Sather's -- some minor details,
> such as the fact that to get the next item from a Python iterator
> you call .next() on it (or use it in the headclause of a for
> statement), while a Sather iterator gives its next value whenever
> you use it within a loop statement, are little more than syntax
> sugar (Sather's sugar is nicer, Python's way simpler to retrofit
> into an existing language;-).

Some editors could convert '!' to '.next()'.  One would just have to
be careful in comments and strings ;<), unless the editor was also
code/comment/string-literal aware.

> Sather's iterators are _way_ more powerful than Python's because
> Sather's can accept both "hot" arguments (evaluated anew each time
> the iterator is called) and "once" ones (evaluated only at the first
> call on the iterator).

Python generators can be called with mutable arguments, can they not?
This is somewhat analogous to using mutable defaults and even 'hotter'
(more bug and confusion prone) since arg mutations are not confined to
the body of the function or even calls thereto.

Without discipline, this is a poor way to write a consumer iterator.
Reduce() is Python's generic consumer iterator.  Python's slice
setting generalizes Sather's set!() and it therefore arguably better
even though a special syntax is needed.  Any class can be given
whatever iterator-like consumer methods one wants to write.

I see four main differences between a consumer function and a (Sather)
consumer iterator:
1. The iterator is called n times instead of once (bad) but the
function may make n calls if it gets an producer iterator rather than
an explicit sequence (then even).
2. The iterator returns n results (n-1 partial) instead of just the
final one (bad unless they are actually wanted, in which case the
function can return a list).
3. The iterator can yield and resume anywhere in the body text, though
the example given yields as the bottom.  However, a function can just
as easily grab the next item via indexing or .next() call anywhere it
pleases, as needed.  Seems pretty even to me.
4. The iterator maintains state in local variables between calls while
a function written to act as an iterator has to either use global
variables (slower and subject to clashes with other processes), use
instance variables (slower but probably the default best choice,
though still subject to clash with other methods of the same
instance), or stash and retrieve local variables in instance variables
(slower and cumbersome, so should be limited to variables used
repeatedly in internal loops; but such may not be needed in functions
meant to be called repeatedly with one item at a time).  This is a win
for Sather's generalization of yield.

>All in all, if a totally new language was being designed I think
>I'd go for something closer to Sather's iterators rather than
>Python's, but its a delicate judgment ...

Each fit into their language.  Sather mostly lies between S-expression
Lisp and Python in terms of  syntactic uniformity/diversity.  Both
uniformity and variety have advantages as others have discussed.  If a
language has non-function statement syntax like 'loop ... end' and 'if
... then ...', then I prefer that control items like while, until, and
break be part of that statement syntax.  I (and others) see while as a
multiple-execution version of if.  On the other hand, resumable
consumers are in general as sensible as resumable producers.  It is
Python's for-loops which make the latter more special for Python.

Terry J. Reedy