[Python-Dev] Obtaining stack-frames from co-routine objects

Ben Leslie benno at benno.id.au
Sun May 31 14:35:57 CEST 2015

Hi Yury,

I'm just starting my exploration into using async/await; all my
'real-world' scenarios are currently hypothetical.

One such hypothetical scenario however is that if I have a server
process running, with some set of concurrent connections, each managed
by a co-routine. Each co-routine is of some arbitrary complexity e.g:
some combination of reading files, reading from database, reading from
peripherals. If I notice one of those co-routines appears stuck and
not making progress, I'd very much like to debug that, and preferably
in a way that doesn't necessarily stop the rest of the server (or even
the co-routine that appears stuck).

The problem with the "if debug: log(...)" approach is that you need
foreknowledge of the fault state occurring; on a busy server you don't
want to just be logging every 'switch()'. I guess you could do
something like "switch_state[outer_coro] = get_current_stack_frames()"
on each switch. To me double book-keeping something that the
interpreter already knows seems somewhat wasteful but maybe it isn't
really too bad.



On 29 May 2015 at 23:57, Yury Selivanov <yselivanov.ml at gmail.com> wrote:
> Hi Ben,
> Is there any real-world scenario where you would need this?
> It looks like this can help with debugging, somehow, but the easiest
> solution is to put a "if debug: log(...)" before "yield" in your
> "switch()" function.  You'll have a perfect traceback there.
> Thanks,
> Yury
> On 2015-05-29 12:46 AM, Ben Leslie wrote:
>> Hi all,
>> Apologies in advance; I'm not a regular, and this may have been
>> handled already (but I couldn't find it when searching).
>> I've been using the new async/await functionality (congrats again to
>> Yury on getting that through!), and I'd like to get a stack trace
>> between the place at which blocking occurs and the outer co-routine.
>> For example, consider this code:
>> """
>> async def a():
>>      await b()
>> async def b():
>>      await switch()
>> @types.coroutine
>> def switch():
>>      yield
>> coro_a = a()
>> coro_a.send(None)
>> """
>> At this point I'd really like to be able to somehow get a stack trace
>> similar to:
>> test.py:2
>> test.py:4
>> test.py:9
>> Using the gi_frame attribute of coro_a, I can get the line number of
>> the outer frame (e.g.: line 2), but from there there is no way to
>> descend the stack to reach the actual yield point.
>> I thought that perhaps the switch() co-routine could yield the frame
>> object returned from inspect.currentframe(), however once that
>> function yields that frame object has f_back changed to None.
>> A hypothetical approach would be to work the way down form the
>> outer-frame, but that requires getting access to the co-routine object
>> that the outer-frame is currently await-ing. Some hypothetical code
>> could be:
>> """
>> def show(coro):
>>      print("{}:{}".format(coro.gi_frame.f_code.co_filename,
>> coro.gi_frame.f_lineno))
>>      if dis.opname[coro.gi_code.co_code[coro.gi_frame.f_lasti + 1]] ==
>>          show(coro.gi_frame.f_stack[0])
>> """
>> This relies on the fact that an await-ing co-routine will be executing
>> a YIELD_FROM instruction. The above code uses a completely
>> hypothetical 'f_stack' property of frame objects to pull the
>> co-routine object which a co-routine is currently await-ing from the
>> stack. I've implemented a proof-of-concept f_stack property in the
>> frameobject.c just to test out the above code, and it seems to work.
>> With all that, some questions:
>> 1) Does anyone else see value in trying to get the stack-trace down to
>> the actual yield point?
>> 2) Is there a different way of doing it that doesn't require changes
>> to Python internals?
>> 3) Assuming no to #2 is there a better way of getting the information
>> compared to the pretty hacking byte-code/stack inspection?
>> Thanks,
>> Ben
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/yselivanov.ml%40gmail.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/benno%40benno.id.au

More information about the Python-Dev mailing list