Reducing "yield from" overhead in recursive generators
Barry
barry at barrys-emacs.org
Sat Mar 19 13:31:22 EDT 2022
> On 19 Mar 2022, at 03:07, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>
> On 19/03/22 9:40 am, Rathmann wrote:
>> The other challenge/question would be to see if there is a way to implement
>> this basic approach with lower overhead.
>
> My original implementation of yield-from didn't have this problem,
> because it didn't enter any of the intermediate frames -- it just
> ran down a chain of C pointers to find the currently active one.
>
> At some point this was changed, I think so that all the frames
> would show up in the traceback in the event of an exception.
> This was evidently seen as more important than having efficient
> resumption of nested generators.
I would have thought that it would be possible to do both sets of pointer/accounting.
One that supports your original fast implementation and add to that the info to do the
trace back.
Barry
>
> I'm not sure exactly how it works now, but I believe it involves
> re-executing the YIELD_FROM bytecode in each generator down the
> chain whenever a resumption occurs. This is nice and simple, but
> not very efficient.
>
> Maybe another way could be found of preserving the traceback,
> such as temporarily splicing the chain of frames into the call
> stack and then resuming the last one.
>
> --
> Greg
> --
> https://mail.python.org/mailman/listinfo/python-list
>
More information about the Python-list
mailing list