On 27 April 2018 at 21:27, Steven D'Aprano wrote:

Obviously dp() would have to be magic. There's no way that I know of for a Python function to see the source code of its own arguments. I have no idea what sort of deep voodoo would be required to make this work. But if it could work, wow, that would really be useful. And not just for beginners.

If you relax the enhancement to just noting the line where the debug print came from, it doesn't need to be deep compiler magic - the same kind of stack introspection that warnings and tracebacks use would suffice. (Stack introspection to find the caller's module, filename and line number, linecache to actually retrieve the line if we want to print that). Cheers, Nick. P.S. While super() is a *little* magic, it isn't *that* magic - it gets converted from "super()" to "super(name_of_first_param, __class__)". And even that limited bit of magic has proven quirky enough to be a recurring source of irritation when it comes to interpreter maintenance. -- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia

On Fri, Apr 27, 2018 at 6:24 PM Nick Coghlan wrote:

On 27 April 2018 at 21:27, Steven D'Aprano wrote:

...

Obviously dp() would have to be magic. There's no way that I know of for a Python function to see the source code of its own arguments. I have no idea what sort of deep voodoo would be required to make this work. But if it could work, wow, that would really be useful. And not just for beginners.

If you relax the enhancement to just noting the line where the debug print came from, it doesn't need to be deep compiler magic - the same kind of stack introspection that warnings and tracebacks use would suffice. (Stack introspection to find the caller's module, filename and line number, linecache to actually retrieve the line if we want to print that).

I​ spent a bit of time and now there's a dprint https://pypi.org/p/dprint project on PyPI. It uses stack introspection to print out some details. If someone wants to take it for a spin and provide feedback on how it feels, this thread is as good a place as any, I suppose. :) Cheers, Pradyun

Cheers, Nick.

P.S. While super() is a *little* magic, it isn't *that* magic - it gets converted from "super()" to "super(name_of_first_param, __class__)". And even that limited bit of magic has proven quirky enough to be a recurring source of irritation when it comes to interpreter maintenance.

-- Nick Coghlan | ncoghlan@gmail.com | Brisbane, Australia _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- -- Pradyun

On Fri, Apr 27, 2018 at 5:58 AM, Chris Angelico wrote:

On Fri, Apr 27, 2018 at 9:27 PM, Steven D'Aprano wrote: I don't think this needs any specific compiler magic or making 'dp' a reserved name, but it might well be a lot easier to write if there were some compiler features provided to _all_ functions. For instance, column positions are currently available in SyntaxErrors, but not other exceptions:

...

...

...

x = 1 print("spam" + x) Traceback (most recent call last): File "<stdin>", line 1, in <module> TypeError: can only concatenate str (not "int") to str print("spam" : x) File "<stdin>", line 1 print("spam" : x) ^ SyntaxError: invalid syntax

Imagine if the TypeError could show a caret, pointing to the plus sign. That would require that a function store column positions, not just line numbers. I'm not sure how much overhead it would add, nor how much benefit you'd really get from those markers, but it would then be the same mechanic for exception tracebacks and for semi-magical functions like this.

Being able to add carets to tracebacks in general would be quite nice actually. Imagine: Traceback (most recent call last): File "/tmp/blah.py", line 16, in <module> print(foo()) ^^^^^ File "/tmp/blah.py", line 6, in foo return bar(1) + bar(2) ^^^^^^ File "/tmp/blah.py", line 10, in bar return baz(2 * x) / baz(2 * x + 1) ^^^^^^^^^^ File "/tmp/blah.py", line 13, in baz return 1 + 1 / (x - 4) ^^^^^^^^^^^ ZeroDivisionError: division by zero This is how I report error messages in patsy[1], and people seem to appreciate it... it would also help Python catch back up with other languages whose error reporting has gotten much friendlier in recent years (e.g., rust, clang). Threading column numbers through the compiler might be tedious but AFAICT should be straightforward in principle. (Peephole optimizations and similar might be a bit of a puzzle, but you can do pretty crude things like saying new_span_start = min(*old_span_starts); new_span_end = max(*old_span_ends) and still get something that's at least useful, even if not necessarily 100% theoretically accurate.) The runtime overhead would be essentially zero, since this would be a static table that only gets consulted when printing tracebacks, similar to the lineno table. (Tracebacks already preserve f_lasti.) So I think the main issue would be the extra memory in each code object to hold the bytecode offset -> column numbers table. We'd need some actual numbers to judge this for real, but my guess is that the gain in usability+friendliness would be easily worth it for 99% of users, and the other 1% are already plotting how to add options to strip out unnecessary things like type annotations so if it's a problem then this could be another thing for them to add to their list – leave out these tables at -OOO or whatever. -n [1] https://patsy.readthedocs.io/en/latest/overview.html -- Nathaniel J. Smith -- https://vorpus.org

On Sat, Apr 28, 2018 at 7:29 PM, Greg Ewing wrote:

but he sent it in HTML using a proportional font, which spoils the effect!

Uh...? https://vorpus.org/~njs/tmp/monospace.png It looks like my client used "font-family: monospace", maybe yours only understands <pre> or something? Anyway, if anyone else is having trouble viewing it, it seems to have come through correctly in the archives: https://mail.python.org/pipermail/python-ideas/2018-April/050137.html -n -- Nathaniel J. Smith -- https://vorpus.org

Actually, I think I can think of a way to make this work, if we're willing to resurrect some old syntax. On Fri, Apr 27, 2018 at 09:27:34PM +1000, Steven D'Aprano wrote:

I think that this is either a great idea or pointless, depending on what the built-in actually does.

If all it does is literally the debug print function you give:

...

# "debug print": prints and then returns its argument def dp(obj): print(repr(obj)) return obj

then it is just a trivial helper as you say, and in my opinion too trivial to bother making a builtin.

I changed my mind... let's add this as a builtin, under the name debugprint. It is a completely normal, non-magical function, which takes four (not one) arguments: def debugprint(obj, lineno=None, module=None, source=None): out = [] if module is not None: if lineno is None: lineno = "?" out.append(f"Line {lineno} of {module}") if source is not None: out.append(ascii(source)) out.append(f"result {repr(obj)}") print(', '.join(out)) return obj Now let's put all the magic into some syntax. I'm going to suggest resurrecting the `` backtick syntax from Python 2. If that's not visually distinct enough, we could double them: ``expression``. When the compiler sees an expression inside backticks, it grabs the name of the module, the line number, and the expression source, and compiles a call to: debugprint(expression, lineno, module, source) in its place. That's the only magic needed, and since it is entirely at compile-time, all that information should be easily available. (I hope.) If not, then simply replace the missing values with None. If the caller shadows debugprint, it is their responsibility to either give it the correct signature, or not to use the backticks. Since it's just a normal function call, the worst that happens is that a mismatch in arguments gives you a TypeError. Shadowing debugprint would be an easy way to disable backticks on a per-module basis, at runtime. Simply define: def debugprint(obj, *args): return obj and Bob's yer uncle. -- Steve

When I teach decorators, I start with a "logged" decorator example: https://uwpce-pythoncert.github.io/PythonCertDevel/modules/Decorators.html#a... def logged_func(func): def logged(*args, **kwargs): print("Function {} called".format(func.__name__)) if args: print("\twith args: {}".format(args)) if kwargs: print("\twith kwargs: {}".format(kwargs)) result = func(*args, **kwargs) print("\t Result --> {}".format(result)) return result return logged Interestingly, I don't actually use such a thing in any kind of production, but it's could be a way to ackomplish what's been proposed here. As a decorator, we usually would expect to use it with the decoration syntax: @logged def a_new_func(): ... But it would also be used to re-bind a already defined function for testing. using the original example: while (func1() + 2 * func2()) < func3(): could become "logged" by adding: func2 = logged(func2) while (func1() + 2 * func2()) < func3(): I actually like that better than inserting extra code into the line you want to test. And I"m pretty clueless about what yu can to with inspect -- but maybe some more magic could be added in the decorator if you wanted that. -CHB On Fri, Apr 27, 2018 at 6:27 AM, Eric Fahlgren wrote:

I've had a 'dprint' in sitecustomize for years. It clones 'print' and adds a couple of keyword parameters, 'show_stack' and 'depth', which give control over traceback output (walks back up sys._getframe for 'depth' entries). It returns the final argument if there is one, otherwise None. It can be used anywhere and everywhere that builtin print is used, plus anywhere in any expression just passing a single argument.

I thought about replacing standard print with it, but I like the greppability of 'dprint' when it comes time to clean things.

On Fri, Apr 27, 2018 at 6:05 AM, Steven D'Aprano wrote:

...

Actually, I think I can think of a way to make this work, if we're willing to resurrect some old syntax.

On Fri, Apr 27, 2018 at 09:27:34PM +1000, Steven D'Aprano wrote:

...

I think that this is either a great idea or pointless, depending on what the built-in actually does.

If all it does is literally the debug print function you give:

...

# "debug print": prints and then returns its argument def dp(obj): print(repr(obj)) return obj

then it is just a trivial helper as you say, and in my opinion too trivial to bother making a builtin.

I changed my mind... let's add this as a builtin, under the name debugprint. It is a completely normal, non-magical function, which takes four (not one) arguments:

def debugprint(obj, lineno=None, module=None, source=None): out = [] if module is not None: if lineno is None: lineno = "?" out.append(f"Line {lineno} of {module}") if source is not None: out.append(ascii(source)) out.append(f"result {repr(obj)}") print(', '.join(out)) return obj

Now let's put all the magic into some syntax. I'm going to suggest resurrecting the `` backtick syntax from Python 2. If that's not visually distinct enough, we could double them: ``expression``.

When the compiler sees an expression inside backticks, it grabs the name of the module, the line number, and the expression source, and compiles a call to:

debugprint(expression, lineno, module, source)

in its place. That's the only magic needed, and since it is entirely at compile-time, all that information should be easily available. (I hope.) If not, then simply replace the missing values with None.

If the caller shadows debugprint, it is their responsibility to either give it the correct signature, or not to use the backticks. Since it's just a normal function call, the worst that happens is that a mismatch in arguments gives you a TypeError.

Shadowing debugprint would be an easy way to disable backticks on a per-module basis, at runtime. Simply define:

def debugprint(obj, *args): return obj

and Bob's yer uncle.

-- Steve _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/

-- Christopher Barker, Ph.D. Oceanographer Emergency Response Division NOAA/NOS/OR&R (206) 526-6959 voice 7600 Sand Point Way NE (206) 526-6329 fax Seattle, WA 98115 (206) 526-6317 main reception Chris.Barker@noaa.gov

On Sat, Apr 28, 2018 at 4:27 AM, Steven D'Aprano wrote:

On Fri, Apr 27, 2018 at 06:27:44AM -0700, Eric Fahlgren wrote:

...

I've had a 'dprint' in sitecustomize for years. It clones 'print' and adds a couple of keyword parameters, 'show_stack' and 'depth', which give control over traceback output (walks back up sys._getframe for 'depth' entries). It returns the final argument if there is one, otherwise None. It can be used anywhere and everywhere that builtin print is used, plus anywhere in any expression just passing a single argument.

Is this published anywhere?

Sorry, no, it's part of work code, but it's pretty simple stuff. 'get_stack' is a debug-quality stack dumper (my memory failed, now uses inspect instead of the more primitive sys._getframe), used whenever anyone wants to see where they are. The 'all' parameter lets you filter out some stdlib entries. def get_stack(all=False, skip=0, depth=0): stack = inspect.stack()[1:] # Implicitly ignore the frame containing get_stack. stack.reverse() # And make it oldest first. length = len(stack) upper = length - skip lower = (upper - depth) if depth else 0 max_len = 0 for i_frame in range(lower, upper): # Preprocessing loop to format the output frames and # calculate the length of the fields in the output. stack[i_frame] = list(stack[i_frame]) path = _cleaner().clean(stack[i_frame][1]) # Canonical form for paths... if all or '/SITE-P' not in path.upper() and '

On Fri, Apr 27, 2018 at 12:35:22PM -0400, Clint Hepner wrote:

I don't want to hijack the thread on a digression, but instead of bringing `` back for just this one purpose, it could be used as a prefix to define a candidate pool of new keywords.

``debugprint(obj) # instead of ``obj`` meaning debugprint(obj)

Part of the reason for me picking `obj` or ``obj`` is that I want it to be short and easy to type, and not very distracting. Neither of those apply to debugprint. -- Steve