[Cython] Recent bugs in generators

Vitja Makarov vitja.makarov at gmail.com
Wed Apr 20 12:51:35 CEST 2011

2011/4/20 Stefan Behnel <stefan_ml at behnel.de>:
> Vitja Makarov, 20.04.2011 11:50:
>> 2011/4/20 Stefan Behnel:
>>> Vitja Makarov, 20.04.2011 10:26:
>>>> 2011/4/18 Stefan Behnel:
>>>>> generators have their own lifetime frame in CPython, and
>>>>> exceptions don't leak from that. So, whenever it's the generator (or
>>>>> code
>>>>> called by it) that raises an exception, that must be kept local to the
>>>>> generator.
>>>> Please review:
>>>> https://github.com/vitek/cython/commit/73014aaed10b82a3f632d7f86212f86280c55858
>>>> I've added __Pyx_Generator_SwapExceptions() method and call it right
>>>> before resume switch and before return from yield. It swaps generator
>>>> exception state with thread local.
>>> Looks good to me. I assume this fixes the problem? Then please push it
>>> into
>>> mainline.
>> old pull request is still there ;)
>> https://github.com/cython/cython/pull/25
>> Does __Pyx_ExceptionReset() steal references to args, so they should
>> not be decrefed later?
> Hmm, good call. The refcounting looks correct over the two function calls,
> but it would be nicer if it could be avoided instead. We are really just
> swapping around pointers here, no refcounting is needed.
> I think it's worth using a dedicated utility function for this.

Do you mean new utility call __Pyx_ExceptionSwap()?

> Oh, and one more thing: what should be done when exiting the generator
> normally? Would that just raise GeneratorExit anyway, so that we can swallow
> any original outer exception? I think there might be a problem with Python
> 3, where the GeneratorExit should be raised in the context of the original
> exception. Worth testing how CPython behaves here...

Right! It's better to wrap call to generator:


Ok, I'll take a look.


More information about the cython-devel mailing list