Generators and propagation of exceptions

[Terry Reedy]

On 4/8/2011 11:55 AM, r wrote:
> I had a problem for which I've already found a "satisfactory"
> work-around, but I'd like to ask you if there is a better/nicer
> looking solution. Perhaps I'm missing something obvious.
>
> The code looks like this:
>
> stream-of-tokens = token-generator(stream-of-characters)
> stream-of-parsed-expressions = parser-generator(stream-of-tokens)
> stream-of-results = evaluator-generator(stream-of-parsed-expressions)
>
> each of the above functions consumes and implements a generator:
>
> def evaluator-generator(stream-of-tokens):

According to the above, that should be stream-of-parsed-expressions.

>    for token in stream-of-tokens:
>      try:
>         yield token.evaluate()           # evaluate() returns a Result
>      except Exception as exception:
>         yield ErrorResult(exception) # ErrorResult is a subclass of Result

The question which you do not answer below is what, if anything, you 
want to do with error? If nothing, just pass. You are now, in effect, 
treating them the same as normal results (at least sending them down the 
same path), but that does not seem satisfactory to you. If you want them 
treated separately, then send them down a different path. Append the 
error report to a list or queue or send to a consumer generator 
(consumer.send).

> The problem is that, when I use the above mechanism, the errors
> propagate to the output embedded in the data streams. This means, I
> have to make them look like real data (in the example above I'm
> wrapping the exception with an ErrorExpression object) and raise them
> and intercept them again at each level until they finally trickle down
> to the output. It feels a bit like writing in C (checking error codes
> and propagating them to the caller).
>
> OTOH, if I don't intercept the exception inside the loop, it will
> break the for loop and close the generator. So the error no longer
> affects a single token/expression but it kills the whole session. I
> guess that's because the direction flow of control is sort of
> orthogonal to the direction of flow of data.

-- 
Terry Jan Reedy