[Twisted-Python] Strange recursion error with twisted.web
I'm getting a "maximum recursion depth exceeded" error that appears to be coming from flatten(). The odd thing is that it only happens sometimes. The HTML that's being flattened does have a few Deferreds in it. Those come from function calls, which cache the results, which might explain why I only see the error on the first visit to the page (as far as I can tell).
The system recursion limit is the standard 1000. My HTML is only nested a few tags deep, two orders of magnitude short of that. Is there anything about the way flatten() works that might cause this behaviour?
Peter.
On Mar 9, 2021, at 4:54 AM, Peter Westlake peter.westlake@pobox.com wrote:
I'm getting a "maximum recursion depth exceeded" error that appears to be coming from flatten(). The odd thing is that it only happens sometimes. The HTML that's being flattened does have a few Deferreds in it. Those come from function calls, which cache the results, which might explain why I only see the error on the first visit to the page (as far as I can tell).
The system recursion limit is the standard 1000. My HTML is only nested a few tags deep, two orders of magnitude short of that. Is there anything about the way flatten() works that might cause this behaviour?
flatten() can definitely result in some deep recursive stacks, particularly in combination with synchronous Deferreds which have their own accumulating stack costs. I'd be interested to see a minimal reproducer for this though, I'm sure we could do a lot better.
-g
On Tue, 9 Mar 2021, at 19:28, Glyph wrote:
On Mar 9, 2021, at 4:54 AM, Peter Westlake peter.westlake@pobox.com wrote:
I'm getting a "maximum recursion depth exceeded" error that appears to be coming from flatten(). The odd thing is that it only happens sometimes.
flatten() can definitely result in some deep recursive stacks, particularly in combination with synchronous Deferreds which have their own accumulating stack costs. I'd be interested to see a minimal reproducer for this though, I'm sure we could do a lot better.
It’s good to know that this isn’t completely unexpected. I’ll try to reproduce it, thanks.
Peter.
On Tue, 9 Mar 2021, at 19:28, Glyph wrote:
On Mar 9, 2021, at 4:54 AM, Peter Westlake peter.westlake@pobox.com wrote:
I'm getting a "maximum recursion depth exceeded" error that appears to be coming from flatten(). The odd thing is that it only happens sometimes. The HTML that's being flattened does have a few Deferreds in it. Those come from function calls, which cache the results, which might explain why I only see the error on the first visit to the page (as far as I can tell).
The system recursion limit is the standard 1000. My HTML is only nested a few tags deep, two orders of magnitude short of that. Is there anything about the way flatten() works that might cause this behaviour?
flatten() can definitely result in some deep recursive stacks, particularly in combination with synchronous Deferreds which have their own accumulating stack costs. I'd be interested to see a minimal reproducer for this though, I'm sure we could do a lot better.
Here it is:
import sys from twisted.internet import reactor, defer, task from twisted.web.template import flatten
def output(stuff): sys.stdout.write(stuff.decode())
def sync(reactor): return flatten(None, [defer.succeed(str(i)+'\n') for i in range(1000)], output)
task.react(sync)
It fails after printing 197 lines. The same sort of thing using deferLater instead of defer.succeed printed 1000 without error.
Peter.
On Mar 10, 2021, at 3:09 AM, Peter Westlake peter.westlake@pobox.com wrote:
On Tue, 9 Mar 2021, at 19:28, Glyph wrote:
On Mar 9, 2021, at 4:54 AM, Peter Westlake peter.westlake@pobox.com wrote:
I'm getting a "maximum recursion depth exceeded" error that appears to be coming from flatten(). The odd thing is that it only happens sometimes. The HTML that's being flattened does have a few Deferreds in it. Those come from function calls, which cache the results, which might explain why I only see the error on the first visit to the page (as far as I can tell).
The system recursion limit is the standard 1000. My HTML is only nested a few tags deep, two orders of magnitude short of that. Is there anything about the way flatten() works that might cause this behaviour?
flatten() can definitely result in some deep recursive stacks, particularly in combination with synchronous Deferreds which have their own accumulating stack costs. I'd be interested to see a minimal reproducer for this though, I'm sure we could do a lot better.
Here it is:
import sys from twisted.internet import reactor, defer, task from twisted.web.template import flatten
def output(stuff): sys.stdout.write(stuff.decode())
def sync(reactor): return flatten(None, [defer.succeed(str(i)+'\n') for i in range(1000)], output)
task.react(sync)
It fails after printing 197 lines. The same sort of thing using deferLater instead of defer.succeed printed 1000 without error.
Peter.
Would you mind filing a ticket in trac, or digging one up if you can find it? This problem rings a bell, and I think I might actually have some code for this lying around somewhere already.
-g
On Wed, 10 Mar 2021, at 23:11, Glyph wrote:
Would you mind filing a ticket in trac, or digging one up if you can find it? This problem rings a bell, and I think I might actually have some code for this lying around somewhere already.
Here is it: https://twistedmatrix.com/trac/ticket/10125#ticket
thanks,
Peter.
On Mar 11, 2021, at 4:12 AM, Peter Westlake peter.westlake@pobox.com wrote:
On Wed, 10 Mar 2021, at 23:11, Glyph wrote:
Would you mind filing a ticket in trac, or digging one up if you can find it? This problem rings a bell, and I think I might actually have some code for this lying around somewhere already.
Here is it: https://twistedmatrix.com/trac/ticket/10125#ticket
thanks,
Peter.
I did indeed have a half-finished experiment to fix this. The fix is now finished and in review here: https://twistedmatrix.com/trac/ticket/10125#comment:1 https://twistedmatrix.com/trac/ticket/10125#comment:1 .
If you could review it thoroughly enough, I could land it.
-g
On Sun, 14 Mar 2021, at 02:08, Glyph wrote:
I did indeed have a half-finished experiment to fix this. The fix is now finished and in review here: https://twistedmatrix.com/trac/ticket/10125#comment:1 .
If you could review it thoroughly enough, I could land it.
Excellent, thank you!
I think I understand how it works, but one thing puzzles me: how does the queue in the jump() function ever get longer than one element? Is it because it might contain a recursive call, which is activated by the unpause()?
Peter.
On Mar 16, 2021, at 4:20 AM, Peter Westlake peter.westlake@pobox.com wrote:
On Sun, 14 Mar 2021, at 02:08, Glyph wrote:
I did indeed have a half-finished experiment to fix this. The fix is now finished and in review here: https://twistedmatrix.com/trac/ticket/10125#comment:1 https://twistedmatrix.com/trac/ticket/10125#comment:1 .
If you could review it thoroughly enough, I could land it.
Excellent, thank you!
I think I understand how it works, but one thing puzzles me: how does the queue in the jump() function ever get longer than one element? Is it because it might contain a recursive call, which is activated by the unpause()?
This was actually a really insightful review - after thinking about it for a moment, I realized that the actual trampoline was already in twisted.internet.defer._inlineCallbacks, and this function was totally useless complexity! Thank you!
-g
On Wed, 17 Mar 2021, at 02:50, Glyph wrote:
Excellent, thank you!
I think I understand how it works, but one thing puzzles me: how does the queue in the jump() function ever get longer than one element? Is it because it might contain a recursive call, which is activated by the unpause()?
This was actually a really insightful review - after thinking about it for a moment, I realized that the *actual* trampoline was already in twisted.internet.defer._inlineCallbacks, and this function was totally useless complexity! Thank you!
Glad to be of service :-)
Hello Twisted team,
Do we have any idea when the fix for "RecursionError with synchronous deferreds (#1549)" will appear in a release?
Peter.
On Mon, Jun 14, 2021 at 05:25:42PM +0100, Peter Westlake wrote:
Do we have any idea when the fix for "RecursionError with synchronous deferreds (#1549)" will appear in a release?
It's been merged to trunk, so it'll be in the next release... whenever that is. Twisted tends to release about once or twice a year but I don't think there's a schedule.
Are you able to test against a recent development version? It's rare for trunk to have regressions, so it should be pretty safe to run against a recent version if it fixes a crash you're having.
On Tue, 15 Jun 2021, at 01:58, Wim Lewis wrote:
On Mon, Jun 14, 2021 at 05:25:42PM +0100, Peter Westlake wrote:
Do we have any idea when the fix for "RecursionError with synchronous deferreds (#1549)" will appear in a release?
It's been merged to trunk, so it'll be in the next release... whenever that is. Twisted tends to release about once or twice a year but I don't think there's a schedule.
Are you able to test against a recent development version? It's rare for trunk to have regressions, so it should be pretty safe to run against a recent version if it fixes a crash you're having.
I can certainly try. Are development versions available for installation, or do I need to build from source?
Peter.
On Tue, Jun 15, 2021, at 2:46 AM, Peter Westlake wrote:
On Tue, 15 Jun 2021, at 01:58, Wim Lewis wrote:
On Mon, Jun 14, 2021 at 05:25:42PM +0100, Peter Westlake wrote:
Do we have any idea when the fix for "RecursionError with synchronous deferreds (#1549)" will appear in a release?
It's been merged to trunk, so it'll be in the next release... whenever that is. Twisted tends to release about once or twice a year but I don't think there's a schedule.
Are you able to test against a recent development version? It's rare for trunk to have regressions, so it should be pretty safe to run against a recent version if it fixes a crash you're having.
I can certainly try. Are development versions available for installation, or do I need to build from source?
We don't publish snapshots, but you can install directly from GitHub, like:
pip install https://github.com/twisted/twisted/archive/trunk.zip
It's probably wise to specify a commit rather than reference trunk. For example, to install the latest commit as of now:
pip install https://github.com/twisted/twisted/archive/4edc214ce9ca61950dd613b4d2dbb6fa8...
Pip can also install from a Git URL, but using the archive URL is faster because you avoid doing a full clone.
---Tom
Works a treat! Thank you!
Peter.
On Tue, 15 Jun 2021, at 18:51, Tom Most wrote:
On Tue, Jun 15, 2021, at 2:46 AM, Peter Westlake wrote:
On Tue, 15 Jun 2021, at 01:58, Wim Lewis wrote:
On Mon, Jun 14, 2021 at 05:25:42PM +0100, Peter Westlake wrote:
Do we have any idea when the fix for "RecursionError with synchronous deferreds (#1549)" will appear in a release?
It's been merged to trunk, so it'll be in the next release... whenever that is. Twisted tends to release about once or twice a year but I don't think there's a schedule.
Are you able to test against a recent development version? It's rare for trunk to have regressions, so it should be pretty safe to run against a recent version if it fixes a crash you're having.
I can certainly try. Are development versions available for installation, or do I need to build from source?
We don't publish snapshots, but you can install directly from GitHub, like:
pip install https://github.com/twisted/twisted/archive/trunk.zipIt's probably wise to specify a commit rather than reference trunk. For example, to install the latest commit as of now:
pip installhttps://github.com/twisted/twisted/archive/4edc214ce9ca61950dd613b4d2dbb6fa8...
Pip can also install from a Git URL, but using the archive URL is faster because you avoid doing a full clone.
---Tom
Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
And just to be clear, yes, it fixes my bug too.
Peter.
On Wed, 16 Jun 2021, at 12:30, Peter Westlake wrote:
Works a treat! Thank you!
Peter.
On Tue, 15 Jun 2021, at 18:51, Tom Most wrote:
On Tue, Jun 15, 2021, at 2:46 AM, Peter Westlake wrote:
On Tue, 15 Jun 2021, at 01:58, Wim Lewis wrote:
On Mon, Jun 14, 2021 at 05:25:42PM +0100, Peter Westlake wrote:
Do we have any idea when the fix for "RecursionError with synchronous deferreds (#1549)" will appear in a release?
It's been merged to trunk, so it'll be in the next release... whenever that is. Twisted tends to release about once or twice a year but I don't think there's a schedule.
Are you able to test against a recent development version? It's rare for trunk to have regressions, so it should be pretty safe to run against a recent version if it fixes a crash you're having.
I can certainly try. Are development versions available for installation, or do I need to build from source?
We don't publish snapshots, but you can install directly from GitHub, like:
pip install https://github.com/twisted/twisted/archive/trunk.zipIt's probably wise to specify a commit rather than reference trunk. For example, to install the latest commit as of now:
pip installhttps://github.com/twisted/twisted/archive/4edc214ce9ca61950dd613b4d2dbb6fa8...
Pip can also install from a Git URL, but using the archive URL is faster because you avoid doing a full clone.
---Tom
Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python
participants (4)
-
Glyph -
Peter Westlake -
Tom Most -
Wim Lewis