in search of graceful co-routines

[Terry Reedy]

On 5/17/2011 1:04 PM, Chris Withers wrote:
> Hi All,
>
> I'm looking for a graceful pattern for the situation where I have a
> provider of a sequence, the consumer of a sequence and code to moderate
> the two, and where I'd like to consumer to be able to signal to the
> provider that it hasn't succeeded in processing one element in the queue.
>
> So, I'd want the controlling code to look a lot like:
>
> for item in provider:
> try:
> consumer.handleItem(self)
> except:
> provider.failed(item)
>
> Now, since the sequence is long, and comes from a file, I wanted the
> provider to be an iterator, so it occurred to me I could try and use the
> new 2-way generator communication to solve the "communicate back with
> the provider", with something like:
>
> for item in provider:
> try:
> consumer.handleItem(self)
> except:
> provider.send('fail')
> else:
> provider.send('succeed')
>
> ..but of course, this won't work, as 'send' causes the provider
> iteration to continue and then returns a value itself. That feels weird
> and wrong to me, but I guess my use case might not be what was intended
> for the send method.
>
> Anyway, I wonder how other people would write this?
> (I'm particularly interested in a sane way to use the two way
> communication that PEP 342 introduced)

By default, Python iterators operate in pull mode -- consumers request a 
new item when they want one. I believe .send was mostly intended to 
reverse that, to operate in push mode where producers .send() a item to 
a consumer when they are ready to. That is certainly true of examples I 
have seen.

Using .send for feedback to a provider is trickier, as the two other 
posts have shown. Another option is to write an iterator class instead 
of generator function. You can then give the provider a message receive 
method (.send or whatever) that is decoupled from the send-next method. 
Your example above then *would* work, as 'send' would not cause anything 
in the provider except what you want it to cause.

Standard generator functions abbreviate standard iterator classes. When 
one moves beyond standard iterator behavior, the equivalence tends to 
break down.

-- 
Terry Jan Reedy