Hi Mark,

Thanks for your useful feedback. Some comments:

> 1. Errors spanning multiple lines.

That is addressed. Check the latest version of the PEP: we are propising storing the end lines as well:

https://www.python.org/dev/peps/pep-0657/#specification

> 2. Repeated binary operations on the same line.

It has been suggested to add this on top of the previous information, but we think this is problematic since these locations need to be manually calculated in the compiler, while the ranges are derived from the AST ranges. Given how error prone the old manual calculations of the AST were, we really want to avoid manual calculations here. Additionally, many users have mentioned that a single caret is difficult to read and that motivates the ranges in SyntaxErrors that we introduced.

> 3. Tracking locations in the compiler. PEP 657 proposes that no compression be used

No, PEP 657 doesn't specify the compression, which is different (check the latest version). We say:

"The internal storage, compression and encoding of the information is left as an implementation detail and can be changed at any point as long as the public API remains unchanged."

> I think it would be better to provide an API like PEP 626 and not

restrict the internal format used.

Indeed, that is what we are doing. Check https://www.python.org/dev/peps/pep-0657/#id10

Cheers,

Pablo Galindo Salgado

On Mon, 17 May 2021 at 14:17, Mark Shannon <mark@hotpy.org> wrote:

Hi everyone,

I fully agree with the rationale for the PEP, better error messages
always help.
However, I think the proposed implementation could be misleading in some
cases and wastes memory.

Use one position, not one-and-a-half positions.
-----------------------------------------------

The main problem with PEP 657, IMO, is that it wants to present the
location of an error as a pair of positions, but without the line number
of the second position.
Consequently it ends up with one and a half positions, not two.
This does not work well for a number of reasons.

1. Errors spanning multiple lines.

Consider:

(i1 + i2 +
s1
)

Where i1, i2 are integers and s1 is a string.

With a single location, the error points to the second `+`. PEP 657
would highlight the whole line, `i1 + i2 +`, making it unclear where the
error occurred.

2. Repeated binary operations on the same line.

A single location can also be clearer when all the code is on one line.

i1 + i2 + s1

PEP 657:

i1 + i2 + s1
^^^^^^^^^^^^

Using a single location:

i1 + i2 + s1
^

3. Tracking locations in the compiler.

While nicer locations for errors is great, it won't be popular if it has
a negative impact on performance.
Locations need to tracked through the compiler. The simpler the
location, the easier this is to do correctly without a negative
performance impact.
It is already tricky to do this correctly with just line numbers because
we have both source that compiles to no bytecodes (try, pass) and
bytecodes that have no source (implicit return None and except cleanups).

A single location can still be presented as a whole token, as tokenizing
a single line of code is easy and fast. So when presenting an error, the
whole token can be highlighted.

E.g:

NameError
name
^^^^

Compression
-----------

PEP 657 proposes that no compression be used, but also mandates the use
of a lossy compression scheme (truncating all offsets over 255).
I think it would be better to provide an API like PEP 626 and not
restrict the internal format used. In fact, extending or replacing the
API of PEP 626 seems the best approach to me.

I wouldn't worry about memory consumption.
The code object layout and unmarshalling process needs to be
re-implemented for reduced memory use and faster startup:
https://github.com/markshannon/faster-cpython/blob/master/tiers.md#tier-0
A few bytes more or less in the location table(s) is inconsequential.

Cheers,
Mark.
_______________________________________________
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-leave@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KR7ACFCUNMHT4M7R4XNHGRFV27HZBDFD/
Code of Conduct: http://python.org/psf/codeofconduct/