Add something to `yield from` to allow delegating to an already started generator

Currently, there is no way to correctly delegate to a partially consumed generator with `yield from` without re-implementing the entire semantics of `yield from` in user code. (In particular, I'm referring to full-fledged bidirectional generators that make use of .send(), .throw(), and .close().) This is because `yield from` always `send`s `None` as the first value—and this is consistent with how freshly created generator iterators work, but if the generator iterator that is being yielded from has already been partially consumed, one might need to start the `yield from` with a specific value. Here I propose to add something to the language/stdlib to achieve this (without the user having to re-implement `yield from`). Let me illustrate what I mean. Suppose we have an arbitrary "bidirectional" generator function `subgen`, and we want to wrap this in another generator `gen`. The purpose of `gen` is to intercept the first yielded value of `subgen` and delegate the rest of the iteration/generation to the sub-generator. The first thing that might come to mind could be this: ```python def gen(): g = subgen() first = next(g) # do something with first... yield "intercepted" # delegate the rest yield from g ``` But this is wrong, because when the caller `.send`s something back to the generator after getting the first value, it will end up as the value of the `yield "intercepted"` expression, which is ignored, and instead `g` will receive `None` as the first `.send` value, as part of the semantics of `yield from`. So we might think to do this: ```python def gen(): g = subgen() first = next(g) # do something with first... received = yield "intercepted" g.send(received) # delegate the rest yield from g ``` But what we've done here is just moving the problem back by one step: as soon as we call `g.send(received)`, the generator resumes its execution and doesn't stop until it reaches the next yield statement, whose value becomes the return value of the `.send` call. So we'd also have to intercept that and re-send it. And then send *that*, and *that* again, and so on... So this won't do. At this point it is clear that no number of manual `yield` and `send`s will solve this, because `yield from` will always send `None` as the first value, while instead we would like to send in whatever the caller sent last (at the last `yield` expression). So there is no way to use the `yield from` syntax directly; one has to implement its semantics anew with something like this: ```python def adaptor(generator, init_send_value=None): send = init_send_value try: while True: send = yield generator.send(send) except StopIteration as e: return e.value ``` (And then `yield from adaptor(g, init_send_value=received)`.) This is not even a good implementation because it's missing handling of .throw(), .close(), etc. So my proposal is to either add a piece of syntax to `yield from` so that one can specify a starting value other than `None`, e.g. ```python yield from <generator iterator> with <initial value> ``` or add a standard library function like `adaptor` above (but correctly implemented).
participants (1)
-
Paolo Lammens