Just for sake of completeness: def print_json_response(resp, request): request.write(json.dumps(resp)) request.close() It is this close function which is causing the issue I suspect that somehow the fact that the client has closed the connection, is not being handled. On Sun, Aug 4, 2019 at 11:26 PM Waqar Khan <wk80333@gmail.com> wrote:
Hi Glyph, Here is the minimal version
class FooResource(resource.Resource): def render_GET(request): future = asyncio.ensure_future(self.fetch_response(request)) // some async await functions d = Deferred.fromFuture(future) d.addCallback(print_json_response, request) // this is actually where the error is triggered. d.addErrback(lambda failure: failure.trap(defer.CancelledError)) finished_errback = request.notifyFinish() finished_errback.addErrback(self.handle_cancel, d) // simple logs and cancels d by d.cancel() return NOT_DONE_YET
The traceback is like 'Traceback (most recent call last):\n File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/events.py", line 88, in _run\n self._context.run(self._callback, *self._args)\n File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 827, in adapt\n adapt.actual.callback(extracted)\n File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 460, in callback\n self._startRunCallbacks(result)\n File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks\n self._runCallbacks()\n--- <exception caught here> ---\n File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks\n current.result = callback(current.result, *args, **kw)\n File "/Users/wqKhan/twisted_eg/parse_response.py", line 3, in print_json_response\n *print_json_response*(response, request)\n File "/Users/wqKhan/twisted_eg/parse_response.py", line 21, in *print_json_response*\n request.finish()\n File "/usr/local/lib/python3.7/site-packages/twisted/web/server.py", line 268, in finish\n return http.Request.finish(self)\n File "/usr/local/lib/python3.7/site-packages/twisted/web/http.py", line 1071, in finish\n "Request.finish called on a request after its connection was lost; "\nbuiltins.RuntimeError: Request.finish called on a request after its connection was lost; use Request.notifyFinish to keep track of this.\n'
On Sun, Aug 4, 2019 at 10:36 PM Glyph <glyph@twistedmatrix.com> wrote:
On Aug 4, 2019, at 5:57 PM, Waqar Khan <wk80333@gmail.com> wrote:
Hi Glyph, Thanks for the suggestion. I tried the suggestion.. While it fixes the self.channel NoneType issue.. It creates another issue.
Traceback (most recent call last): File "/usr/local/Cellar/python/3.7.1/Frameworks/Python.framework/Versions/3.7/lib/python3.7/asyncio/events.py", line 88, in _run self._context.run(self._callback, *self._args) File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 827, in adapt adapt.actual.callback(extracted) File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 460, in callback self._startRunCallbacks(result) File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 568, in _startRunCallbacks self._runCallbacks() --- <exception caught here> --- File "/usr/local/lib/python3.7/site-packages/twisted/internet/defer.py", line 654, in _runCallbacks current.result = callback(current.result, *args, **kw) ...... request.finish() File "/usr/local/lib/python3.7/site-packages/twisted/web/server.py", line 268, in finish return http.Request.finish(self) File "/usr/local/lib/python3.7/site-packages/twisted/web/http.py", line 1071, in finish "Request.finish called on a request after its connection was lost; " builtins.RuntimeError: Request.finish called on a request after its connection was lost; use Request.notifyFinish to keep track of this.
Oh, hrm. The idea here is that any code that returns NOT_DONE_YET (i.e. does asynchronous work to generate a response) should also be tracking whether it needs to call `.finish()` by watching a `.notifyFinish()` Deferred. I bet that there are plenty of resources in Twisted which don't follow this rule, and we should fix those; but possibly we should also make this error message less stern. Do you have a minimal example?
I have got some handle on the issue. I was wondering if you have any advise on how to debug the issue. Somewhere in the codebase are async/await code (from asyncio world). Hence, when the connection is closed down by client, seems like those pending coroutines are lingering on.
This, I have no idea about. What tasks are you starting, when are you expecting them to get cleaned up, what are they blocking on, how do these interact with Twisted? There are a couple dozen questions I'd have to know the answer to in order to even begin debugging this. If you cancel all the outstanding tasks and look at their tracebacks when exiting it might give you more of a sense of where they're stuck...
-g
_______________________________________________ Twisted-Python mailing list Twisted-Python@twistedmatrix.com https://twistedmatrix.com/cgi-bin/mailman/listinfo/twisted-python