[Python-ideas] Standard way to get caller name and easy call stack access

[anatoly techtonik]

On Thu, Mar 22, 2012 at 11:01 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
> Trying to find out your calling function is hard because well-designed
> decoupled code with properly separated concerns *should never care who
> is calling it*, so improving the APIs for doing so is an *extremely
> low priority task*.

You're saying that somebody specifically designed this part to be
hard, because those who program properly should never request the name
of calling function. Is that what you're wanted to say?

Correct me, but I think you were trying to say that nobody is
interested in improving the way to get caller function, because people
should not write the code that requests it. There are two counter
arguments:
1. De facto existence of the code in standard library that is used a
lot and that makes Python programs slow - logging
2. Practicality that beats purity. Debugging and introspection
capabilities of the language are as important for development and
maintenance as writing well-designed decoupled code with properly
separated concerns (which mostly exists in ideal world without time,
readability and performance constraints).

For people who study Python applications programming that goes beyond
function reference most of the time is spent in reverse coding -
debugging and troubleshooting, and accessing a call stack without
unnecessary implications makes a big difference in understanding how
application (and Python) works.

> There are some truly exceptional cases where it matters. For those,
> the onus is on the developers *implementing those exceptional cases*
> to figure out how to do this in a platform independent way (which may
> involve conditional code that does different things depending on the
> platform). These legitimate use cases are simply too rare for it to be
> worthwhile for the various implementations to agree on a standard way
> to expose the functionality and promise never to change it.

Is there an objective statistics how much a typical application
accesses call stack information directly or indirectly? I have a
Python checkout with its test suite - is there a way to patch test
suite to count call stack accesses?

As I am not a C expert - I can't immediately see any obvious problems
with caching call stack information with qualified names on any
platform. Straightforward idea is to annotate bytecode in a separate
memory page during compilation.

> If all you are trying to do is print some debugging info and you can
> use Python 3.2, then use the "stack_info=True" argument to the logging
> APIs. That's what it's for. If you can't use Python 3.2, then look at
> the way logging does it in 3.2 and see if you can adapt that for your
> own needs.

I could respond with "In the face of ambiguity, refuse the temptation
to guess.", but you're right - my original intention was to get the
name of the caller for writing debug info into a file (without logging
module though as seemed to be a reason for deadlock I was tying to
debug). But when I saw how hard is it to understand Python call stack
from the code, how complicated is to get to it (and how much time I
spent to get what I need), and it the end information about Python
caller is _still incomplete_ - I thought about other implications that
this have on Python development as a language in a broader sense.

There is a clojure-py project, Lisp written in Python, RPython and
many other things, but understanding of code differences of these
languages is obscured by the fact that the basic level in Python (on
top of which all these things are written) is too low level. Python
became a platform, and it would be nice when a platform is transparent
and offers convenient debug capabilities (I tend to do Python
development on Fedora, because it provides nice Python debugging
commands in gdb).

> There's no compelling reason that writing helpers for debugging call
> stacks needs to be easy. It should definitely be *possible*, but
> 99.99% of all Python programmers are just going to use the native
> debugging facilities of the interpreter, or use those of an IDE that
> someone else wrote.

Debugging facilities of the interpreter.. Huh.
inspect.stack()[2][0].f_locals['self'].__class__.__name__ - is that
the thing you keep in your head when breaking into Python console to
get a name of a class for a calling method?
And what should be invoked if a caller is not a method?

I find it convenient to quickly insert print statements (or probes) at
the points of interest to explore run-time behavior at these points
instead of inserting awkward logging imports and formatting calls all
over the place. I also don't know which IDE can better help with
debugging multi-threaded application in Python with Qt bindings. So I
just want to have native debugging capabilities of the interpreter to
be convenient and actually useful.

> Once again, you're painting with a broad sweeping brush "ah, it's
> horrible, it's too hard to do anything worthwhile" (despite the
> obvious contradiction that many projects already implement such things
> quite effectively) instead of suggesting *small*, *incremental*
> changes that might actually improve the status quo.

Well, I proposed to have a caller_name() method. If it is not too
universal, then a sys.call_stack() could be a better try:

  sys.call_stack() - Get current call stack. Return the list with a
                            list of qualified names starting with the oldest.

Rationale: make logging calls and module based filters zero cost.

-- 
anatoly t.