Re: [pypy-dev] Mysterious IndexError in service running with PyPy
Hi,
On 22 May 2018 at 04:46, hubo
I think I found the cause, happily it is not a PyPy bug (although it is PyPy-related). Some generators are calling some cleanup code in the "finally" clause, but they are not correctly collected in PyPy until GC (because PyPy does not use reference counting). Apparently when they are collected by GC module, the "finally" clause is executed in a separated thread.
Unlike Java, we're not purposefully running finalizers in its own threads: instead, if your program has got more than one thread running, finalizers will run in the one where the GC happens to run. There isn't much we can do about that. All these hard GC issues also exist on CPython, but they are just more hidden. Typically, if there is a cycle of objects which itself holds, indirectly, a reference to the generator-iterator, then you get the same effect: the cycle will be broken by the GC module of CPython at a random point in time and from a random thread, too. So we can argue that you potentially have a bug even on CPython, in the sense that an apparently unrelated change somewhere else can make the same bug appear on CPython. A bientôt, Armin.
Thank you for your explaination!
One more thing I would like to confirm: looks like in PyPy3, a "yield from my_generator()" expression delegates all exceptions inside, so the finalizer in my_generator() is always executed. Am I right?
2018-05-22
hubo
发件人：Armin Rigo
I think I found the cause, happily it is not a PyPy bug (although it is PyPy-related). Some generators are calling some cleanup code in the "finally" clause, but they are not correctly collected in PyPy until GC (because PyPy does not use reference counting). Apparently when they are collected by GC module, the "finally" clause is executed in a separated thread.
Unlike Java, we're not purposefully running finalizers in its own threads: instead, if your program has got more than one thread running, finalizers will run in the one where the GC happens to run. There isn't much we can do about that. All these hard GC issues also exist on CPython, but they are just more hidden. Typically, if there is a cycle of objects which itself holds, indirectly, a reference to the generator-iterator, then you get the same effect: the cycle will be broken by the GC module of CPython at a random point in time and from a random thread, too. So we can argue that you potentially have a bug even on CPython, in the sense that an apparently unrelated change somewhere else can make the same bug appear on CPython. A bientôt, Armin.
participants (2)
-
Armin Rigo -
hubo