Reducing "yield from" overhead in recursive generators

[Antoon Pardon]

Op 18/03/2022 om 04:14 schreef Rathmann:
> ...
>
> ```python
> def recursive_generator_driver(g):
>      """g is a generator object, which is likely to call other generators"""
>      stack = [g]
>      while True:
>          g = stack[-1]
>          try:
>              val = next(g)
>              if isinstance(val, types.GeneratorType):
>                  # Yielded value represented a recursive call, so push
>                  stack.append(val)
>              else:
>                  # Regular value for iterator, give to caller
>                  yield val
>          except StopIteration:
>              stack.pop()
>              if len(stack) == 0:
>                  return
> ```
>
> and the modified tree traverser is just
>
> ```python
> def inorder_traverse_mod(node):
>      """ONLY for use with recursive_generator_driver"""
>      k, l, r = node
>      if l:
>          yield inorder_traverse_mod(l) # NOT yield from
>      yield k
>      if r:
>          yield inorder_traverse_mod(r) # NOT yield from
> ```
>
> ...
>
> So what do people think of this hack/technique?  Is it
>
> A) A terrible idea?  (Please be specific.)
> B) Already well-known (and I just missed it.)
> C) Potentially useful for the niche case of deeply recursive
>     generators?

I would go with B' + C. It seems there is a resemblance with the trampoline technique
in order to eliminate tail recursion. I know the people of macropy have written a
decorating macro that would eliminate tail recursion from such decorated functions.
Maybe if you contact them they can be interested in making a similar decorating macro
for use with such recursive decorators.

-- 
Antoon Pardon.