[issue22221] ast.literal_eval confused by coding declarations

[Terry J. Reedy]

Terry J. Reedy added the comment:

This issue is about the SyntaxError message for eval functions, not the ast module per se. My first response is that the reported message is not a bug and that this issue should be closed as 'not a bug'.

(General reason) Trying to eval an expression preceded by a comment on its own line or followed by a comment works.

>>> eval("#before\n'string'#after")
'string'

Trying to eval a bare comment *is* a syntax error.

>>> eval("#comment\n")
...
SyntaxError: unexpected EOF while parsing

So the issue as presented is the special-case message.  However, messages are not part of the language specification and improving them is often/usually/always? treated as an enhancement.  Changing them will break code and tests that depend on the exact wording. 2.7 does not get enhancements.

(Specific reason) In 2.x, the input to (literal-)eval is either latin-1 encoded bytes or unicode. 'Latin-1' input could potentially consist of an encoding declaration on one line followed on the next line by a literal string encoded as indicated.

>>> le("# -*- coding: utf-8 -*-\n'string'")
'string'

Unicode input, the subject of this issue, is encoded to latin-1, which means that any literal string in the expression has to be latin-1 encoded. Therefore, a latin-1 encoding declaration is redundant and anything else is either redundant (if the original unicode only contains characters that encode the same in latin-1, as in the example above) or wrong, with hard to predict behavior.  Someone thought it worthwhile to add the special case check.  I think it should be left as is.

Jorgen, please either close this or explain why you think not, in light of the above.

----------
nosy: +terry.reedy

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue22221>
_______________________________________