[Python-ideas] x=(yield from) confusion [was:Yet another alternative name for yield-from]

[Nick Coghlan]

Jim Jewett wrote:
> On 4/3/09, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Greg tried to clarify this a bit already, but I think Jacob's averager
>> example is an interesting case where it makes sense to both yield
>> multiple times and also "return a value".
> 
>>   def averager(start=0):
>>     # averager that maintains a running average
>>     # and returns the final average when done
>>     count = 0
>>     exc = None
>>     sum = start
>>     while 1:
>>       avg = sum / count
>>       try:
>>         val = yield avg
>>       except CalcAverage:
>>         return finally avg
>>       sum += val
>>       count += 1
> 
> It looks to me like it returns (or yields) the running average either way.
> 
> I see a reason to send in a sentinel value, saying "Don't update the
> average, just tell me the current value."
> 
> I don't see why that sentinel has to terminate the generator, nor do I
> see why that final average has to be returned rather than yielded.

It doesn't *have* to do anything - you could set it up that way if you
wanted to. However, when the running average is only yielded, then the
location providing the numbers (i.e. the top level code in my example)
is also the only location which can receive the running average. That's
why anyone using coroutines now *has* to have a top-level scheduler to
handle the "coroutine stack" by knowing which coroutines are calling
each other and detecting when one has "returned" (i.e. yielded a special
sentinel value that the scheduler recognises) so the return value can be
passed back to the calling coroutine.

I hoped to show the advantage of the separate return value with the
difference calculating coroutine: in that example, having the separate
return value makes it easy to identify the values which need to be
returned to a location *other* than the source of the numbers being
averaged. The example goes through some distinct stages:

- top level code sending numbers to first averager via send() and
receiving running averages back via yield
- top level code telling first averager to finish up (via either throw()
or send() depending on implementation), first averager returning final
average value to the difference calculator
- top level code sending numbers to second averager and receiving
running averages back
- top level code telling second averager to finish up, second averager
returning final average value to the difference calculator, difference
calculator returning difference between the two averages to the top
level code

That is, yield, send() and throw() involve communication between the
currently active subcoroutine and the client of the whole coroutine.
They bypass the current stack in the coroutine itself. The return
values, on the other hand, *do* involve unwinding the coroutine stack,
just like they do with normal function calls.

I'll have another go, this time comparing the "normal" calls to the
coroutine version:

  def average(seq, start=0):
    if seq:
      return sum(seq, start) / len(seq)
    return start

  def average_diff(seq1, seq2):
    avg1 = average(seq1)
    avg2 = average(seq2)
    return avg2 - avg1

OK, simple and straightforward. The idea of 'yield from' is to allow
"average_diff" to be turned into a coroutine that receives the values to
be averaged one by one *without* having to inline the actual average
calculation.

I'll also go back to Jacob's original averaging example that *doesn't*
yield a running average - it is more inline with examples where the
final calculation is expensive, so you won't do it until you're told you
have all the data.

What might that look like as 'yield from' style coroutines?

  class CalcAverage(Exception): pass

  def average_cr(start=0):
    count = 0
    exc = None
    sum = start
    while 1:
      try:
        # The yield surrenders control to the code
        # that is supplying the numbers to be
        # averaged
        val = yield
      except CalcAverage:
        # The return finally passes the final
        # average back to the code that was
        # asking for the average (which is NOT
        # necessarily the same code that was
        # supplying the numbers
        if count:
          return finally sum / count
        return finally start
      sum += val
      count += 1

  # Note how similar this is to the normal version above
  def average_diff_cr(start):
    avg1 = yield from average_cr(start, seq1)
    avg2 = yield from average_cr(start, seq2)
    return avg2 - avg1

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------