3.11 enhanced error location - can it be smarter?
"When printing tracebacks, the interpreter will now point to the exact expression that caused the error instead of just the line."
I get the motivation for better error messages, but there are scenarios where you can't provide useful new information. I've been lax in getting to testing the project I work on with 3.11, but here's some spew from one testcase (this is an intentionally triggered error - the testcase is testing predictable behavior in the face of this error, 3.11 didn't *cause* this, as can be fairly easily seen):
Traceback (most recent call last): File "C:\Users\mats\Documents\github\scons\SCons\Script\Main.py", line 1403, in main _exec_main(parser, values) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\mats\Documents\github\scons\SCons\Script\Main.py", line 1366, in _exec_main _main(parser) ^^^^^^^^^^^^^ File "C:\Users\mats\Documents\github\scons\SCons\Script\Main.py", line 1034, in _main SCons.Script._SConscript._SConscript(fs, script) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\mats\Documents\github\scons\SCons\Script\SConscript.py", line 285, in _SConscript exec(compile(scriptdata, scriptname, 'exec'), call_stack[-1].globals) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\mats\AppData\Local\Temp\testcmd.16164.fpv9xt5b\SConstruct", line 4, in <module> raise InternalError('error inside') ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ SCons.Errors.InternalError: error inside
The grump here is all those pointy hats added absolutely zero new information, because in each case the entire line is underlined. So the "benefit" of the change is the traceback went from 12 lines to 17. In many cases tracebacks are much longer and adding lots of not-useful lines is not a great thing.
A thought - how about omitting the underline line if the to-be-underlined part would be the whole line?
On 1/18/22 10:43 AM, Mats Wichmann wrote:
A thought - how about omitting the underline line if the to-be-underlined part would be the whole line?
I would also like that change -- when the underlining is a portion of the whole it's quite useful, but when it's the whole line it's a lot of extra noise.
In fact, if I can be permitted to dream for a moment, what would be really nice is using highlighting or bolding or some such and skipping the extra underline lines (where "underlining" means "extra line with pointy hats").
~Ethan~
We considered using colours and other markers such as bold text, but that opens a considerable can of worms with detecting in all systems and configurations if that can be done. I have been told that some of these situations are quite tricky and is not as easy as checking for tty support.
If someone wants to contribute a way to detect reliably (in C) if the terminal supports ANSI colours, I'm happy to consider switching to that instead of the underlying.
On Tue, 18 Jan 2022, 19:13 Ethan Furman, ethan@stoneleaf.us wrote:
On 1/18/22 10:43 AM, Mats Wichmann wrote:
A thought - how about omitting the underline line if the to-be-underlined part would be the whole line?
I would also like that change -- when the underlining is a portion of the whole it's quite useful, but when it's the whole line it's a lot of extra noise.
In fact, if I can be permitted to dream for a moment, what would be really nice is using highlighting or bolding or some such and skipping the extra underline lines (where "underlining" means "extra line with pointy hats").
~Ethan~ _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/JYC5D3L6... Code of Conduct: http://python.org/psf/codeofconduct/
On 1/18/2022 7:59 PM, Pablo Galindo Salgado wrote:
We considered using colours and other markers such as bold text, but that opens a considerable can of worms with detecting in all systems and configurations if that can be done. I have been told that some of these situations are quite tricky and is not as easy as checking for tty support.
If someone wants to contribute a way to detect reliably (in C) if the terminal supports ANSI colours, I'm happy to consider switching to that instead of the underlying.
I'll save some time - no such mechanism exists on Windows. There's no reliable way to detect what's displaying console output, and while many now support ANSI colours, we can't be assured of it.
Omitting the line of ^ where over x% (75%? 90%?) of characters on the line would be marked would be fine by me.
Cheers, Steve
We cannot base the computation on a % because is possible that the location markers are relevant but the variables, function names or constructs are just very large. I think that the idea of "spawns the whole line" is sensible, though.
On Tue, 18 Jan 2022 at 20:32, Steve Dower steve.dower@python.org wrote:
On 1/18/2022 7:59 PM, Pablo Galindo Salgado wrote:
We considered using colours and other markers such as bold text, but that opens a considerable can of worms with detecting in all systems and configurations if that can be done. I have been told that some of these situations are quite tricky and is not as easy as checking for tty
support.
If someone wants to contribute a way to detect reliably (in C) if the terminal supports ANSI colours, I'm happy to consider switching to that instead of the underlying.
I'll save some time - no such mechanism exists on Windows. There's no reliable way to detect what's displaying console output, and while many now support ANSI colours, we can't be assured of it.
Omitting the line of ^ where over x% (75%? 90%?) of characters on the line would be marked would be fine by me.
Cheers, Steve
On 18/01/2022 20:41, Pablo Galindo Salgado wrote:
We cannot base the computation on a % because is possible that the location markers are relevant but the variables, function names or constructs are just very large. I think that the idea of "spawns the whole line" is sensible, though.
On Tue, 18 Jan 2022 at 20:32, Steve Dower steve.dower@python.org wrote:
Omitting the line of ^ where over x% (75%? 90%?) of characters on the line would be marked would be fine by me.
It would also need to take into account cases where a line contains an inline comment, which does not contribute to the code producing the error, but where all of the rest of the line (the code) is the source of the error and highlighting it is not useful
# test.py:
code_that_causes_an_error # a comment
$ python3.11 test.py
Traceback (most recent call last): File "test.py", line 1, in <module> code_that_causes_an_error # a comment ^^^^^^^^^^^^^^^^^^^^^^^^^ NameError: name 'code_that_causes_an_error' is not defined
(the traceback shown is from a current `main` build)
The code that computes the lines is already quite complex (to the point that has to do some AST analysis and post-parsing) so I am quite worried to introduce a lot of complexity in this area. I am fine doing something that we can easily check for (spawns all the line) but I would be against having to start doing even more analysis (especially for things that are not AST-based).
Take into account that code that displays exceptions need to be very resilient because it can be called in some tricky situations and validating that all the code works correctly is very hard, and complex to maintain.
As I said, I think I would be supportive of considering adding a check for the full line, but I think that adding more complexity here is quite dangerous.
On Tue, 18 Jan 2022 at 21:49, Patrick Reader _@pxeger.com wrote:
On 18/01/2022 20:41, Pablo Galindo Salgado wrote:
We cannot base the computation on a % because is possible that the location markers are relevant but the variables, function names or constructs are just very large. I think that the idea of "spawns the whole line" is sensible, though.
On Tue, 18 Jan 2022 at 20:32, Steve Dower steve.dower@python.org wrote:
Omitting the line of ^ where over x% (75%? 90%?) of characters on the line would be marked would be fine by me.
It would also need to take into account cases where a line contains an inline comment, which does not contribute to the code producing the error, but where all of the rest of the line (the code) is the source of the error and highlighting it is not useful
# test.py:
code_that_causes_an_error # a comment
$ python3.11 test.py
Traceback (most recent call last): File "test.py", line 1, in <module> code_that_causes_an_error # a comment ^^^^^^^^^^^^^^^^^^^^^^^^^ NameError: name 'code_that_causes_an_error' is not defined
(the traceback shown is from a current `main` build) _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/F4FSJY7O... Code of Conduct: http://python.org/psf/codeofconduct/
On 2022-01-18 21:58, Pablo Galindo Salgado wrote:
The code that computes the lines is already quite complex (to the point that has to do some AST analysis and post-parsing) so I am quite worried to introduce a lot of complexity in this area. I am fine doing something that we can easily check for (spawns all the line) but I would be against having to start doing even more analysis (especially for things that are not AST-based).
Take into account that code that displays exceptions need to be very resilient because it can be called in some tricky situations and validating that all the code works correctly is very hard, and complex to maintain.
As I said, I think I would be supportive of considering adding a check for the full line, but I think that adding more complexity here is quite dangerous.
It might be enough to check that there's only whitespace before the first ^ and only whitespace optionally followed by a comment (#) after the last ^ on that line.
On Tue, 18 Jan 2022 at 21:49, Patrick Reader <_@pxeger.com http://pxeger.com> wrote:
On 18/01/2022 20:41, Pablo Galindo Salgado wrote:We cannot base the computation on a % because is possible that the location markers are relevant but the variables, function names or constructs are just very large. I think that the idea of "spawns the whole line" is sensible, though. On Tue, 18 Jan 2022 at 20:32, Steve Dower <steve.dower@python.org <mailto:steve.dower@python.org>> wrote: Omitting the line of ^ where over x% (75%? 90%?) of characters on the line would be marked would be fine by me.It would also need to take into account cases where a line contains an inline comment, which does not contribute to the code producing the error, but where all of the rest of the line (the code) is the source of the error and highlighting it is not useful # test.py: code_that_causes_an_error # a comment $ python3.11 test.py Traceback (most recent call last): File "test.py", line 1, in <module> code_that_causes_an_error # a comment ^^^^^^^^^^^^^^^^^^^^^^^^^ NameError: name 'code_that_causes_an_error' is not defined (the traceback shown is from a current `main` build)
On 1/18/22 11:59 AM, Pablo Galindo Salgado wrote:
We considered using colours and other markers such as bold text, but that opens a considerable can of worms with detecting in all systems and configurations if that can be done. I have been told that some of these situations are quite tricky and is not as easy as checking for tty support.
Maybe use an environment variable?
-- ~Ethan~
tox and pytest look at PY_COLORS:
https://github.com/tox-dev/tox/blob/9cc692d85c9ce84344ea7fee4b127755c6099a32... https://docs.pytest.org/en/6.2.x/reference.html#envvar-PY_COLORS
On Tue, 18 Jan 2022 at 21:17, Ethan Furman ethan@stoneleaf.us wrote:
On 1/18/22 11:59 AM, Pablo Galindo Salgado wrote:
We considered using colours and other markers such as bold text, but
that opens a considerable can of worms with
detecting in all systems and configurations if that can be done. I have
been told that some of these situations are
quite tricky and is not as easy as checking for tty support.
Maybe use an environment variable?
-- ~Ethan~ _______________________________________________ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-leave@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/TRI25YRC... Code of Conduct: http://python.org/psf/codeofconduct/
On 18 Jan 2022, at 19:59, Pablo Galindo Salgado pablogsal@gmail.com wrote:
We considered using colours and other markers such as bold text, but that opens a considerable can of worms with detecting in all systems and configurations if that can be done. I have been told that some of these situations are quite tricky and is not as easy as checking for tty support.
In the apps I work on as open source and paid work tracebacks are put into log files so that we can fix the rare bugs. It would not be nice if the traceback module API started providing text with embedded escape sequences without a way to turn then off in the API.
On the other hand it would be great to be able, as an API call, to set the style of the traceback text.
Barry
On 1/19/22 1:10 PM, Barry Scott wrote:
On 18 Jan 2022, at 19:59, Pablo Galindo Salgado wrote:
We considered using colours and other markers such as bold text, but that opens a considerable can of worms with detecting in all systems and configurations if that can be done. I have been told that some of these situations are quite tricky and is not as easy as checking for tty support.
In the apps I work on as open source and paid work tracebacks are put into log files so that we can fix the rare bugs. It would not be nice if the traceback module API started providing text with embedded escape sequences without a way to turn then off in the API.
An environment variable would solve this, yes? The default would be using the underlining carets, but an env var could switch that to using color instead.
-- ~Ethan~
On 19 Jan 2022, at 21:19, Ethan Furman ethan@stoneleaf.us wrote:
On 1/19/22 1:10 PM, Barry Scott wrote:
On 18 Jan 2022, at 19:59, Pablo Galindo Salgado wrote:
We considered using colours and other markers such as bold text, but that opens a considerable can of worms with detecting in all systems and configurations if that can be done. I have been told that some of these situations are quite tricky and is not as easy as checking for tty support.
In the apps I work on as open source and paid work tracebacks are put into log files so that we can fix the rare bugs. It would not be nice if the traceback module API started providing text with embedded escape sequences without a way to turn then off in the API.
An environment variable would solve this, yes? The default would be using the underlining carets, but an env var could switch that to using color instead.
I have no objection to use of an env var as the default at python startup. But not as the only way to set a feature. I find it a poor API.
Give me a set/get API and I can design my app to behave in a suitable way for its use.
It means that I cannot have all the logic of the app in the python sources.
Barry
On 19 Jan 2022, at 21:19, Ethan Furman ethan@stoneleaf.us wrote:
An environment variable would solve this, yes? The default would be using the underlining carets, but an env var could switch that to using color instead.
Oh and if you use colours then you please give me the ability to set the colours for each usage.
I have custom colour settings for a lots of unix tools so that I get contrast etc. The defaults used may only work in light mode and be unusable dark mode for example.
Barry
On 1/19/2022 5:01 PM, Barry Scott wrote:
Oh and if you use colours then you please give me the ability to set the colours for each usage.
IDLE has that for Error Text, along with other colors.
I have custom colour settings for a lots of unix too ls so that I get contrast etc. The defaults used may only work in light mode and be unusable dark mode for example.
IDLE's color settings come in themes. It comes with customizable light and dark themes.
At the moment, IDLE does not see the extended non-SyntaxError error ranges because the traceback module functions do not provide them by default, or by a function-call argument. But they are supposed to be accessible with some subclassing or monkey-patching, and I intend to do so.
It would not be nice if the traceback module API started providing text with embedded escape sequences without a way to turn then off in the API.
I think fobj.isatty() would give the traceback module a good idea whether it's writing to a display device or not. There are a number of other complications though (APIs, platform differences, TERM environment variables or lack thereof, forcible overriding through an API, what other systems (IDLE, PyCharm, etc) do, ...). If it seems the right place to make a change is in the traceback module, my recommendation would be to fork the existing module and publish your prototype on PyPI. Here's a PyPI module (last updated several years ago) that purports to color traceback output:
https://pypi.org/project/colored-traceback/
(This really belongs on python-ideas, right?)
Skip
On 20 Jan 2022, at 02:22, Skip Montanaro skip.montanaro@gmail.com wrote:
(This really belongs on python-ideas, right?)
I'm commenting on the implementation that is on going. python-ideas does not seem right.
Barry
participants (10)
-
Barney Gale -
Barry Scott -
Ethan Furman -
Mats Wichmann -
MRAB -
Pablo Galindo Salgado -
Patrick Reader -
Skip Montanaro -
Steve Dower -
Terry Reedy