Generators and propagation of exceptions

[Raymond Hettinger]

On Apr 8, 8:55 am, r <nbs.pub... at gmail.com> wrote:
> I had a problem for which I've already found a "satisfactory"
> work-around, but I'd like to ask you if there is a better/nicer
> looking solution. Perhaps I'm missing something obvious.
>
> The code looks like this:
>
> stream-of-tokens = token-generator(stream-of-characters)
> stream-of-parsed-expressions = parser-generator(stream-of-tokens)
> stream-of-results = evaluator-generator(stream-of-parsed-expressions)
>
> each of the above functions consumes and implements a generator:
>
> def evaluator-generator(stream-of-tokens):
>   for token in stream-of-tokens:
>     try:
>        yield token.evaluate()           # evaluate() returns a Result
>     except Exception as exception:
>        yield ErrorResult(exception) # ErrorResult is a subclass of Result
>
> The problem is that, when I use the above mechanism, the errors
> propagate to the output embedded in the data streams. This means, I
> have to make them look like real data (in the example above I'm
> wrapping the exception with an ErrorExpression object) and raise them
> and intercept them again at each level until they finally trickle down
> to the output. It feels a bit like writing in C (checking error codes
> and propagating them to the caller).

You're right, checking and propagating at each level doesn't feel
right.

If you need an exception to break out of the loops and close the
generators, then it seems you should just ignore the exception on
every level except for the one where you really want to handle the
exception (possibly in an outer control-loop):

  while True:
      try:

          for result in evaluator-generator(stream-of-parsed-
expressions):


 let the exception float up to


>
> OTOH, if I don't intercept the exception inside the loop, it will
> break the for loop and close the generator. So the error no longer
> affects a single token/expression but it kills the whole session. I
> guess that's because the direction flow of control is sort of
> orthogonal to the direction of flow of data.
>
> Any idea for a working and elegant solution?
>
> Thanks,
>
> -r