On Tue, Jul 19, 2016 at 8:18 AM Rustom Mody
On Tuesday, July 19, 2016 at 5:06:17 PM UTC+5:30, Neil Girdhar wrote:
On Tue, Jul 19, 2016 at 7:21 AM Steven D'Aprano wrote:
On Mon, Jul 18, 2016 at 10:29:34PM -0700, Rustom Mody wrote:
IOW 1. The lexer is internally (evidently from the error message) so ASCII-oriented that any “unicode-junk” just defaults out to identifiers (presumably comments are dealt with earlier) and then if that lexing action fails it mistakenly pinpoints a wrong *identifier* rather than just an impermissible character like python 2
You seem to be jumping to a rather large conclusion here. Even if you are right that the lexer considers all otherwise-unexpected characters to be part of an identifier, why is that a problem?
It's a problem because those characters could never be part of an identifier. So it seems like a bug.
An armchair-design solution would say: We should give the most appropriate answer for every possible unicode character category This would need to take all the Unicode character-categories and Python lexical-categories and 'cross-product' them — a humongous task to little advantage
I don't see why this is a "humongous task". Anyway, your solution boils down to the simplest fix in the lexer which is to block some characters from matching any category, does it not?
A more practical solution would be to take the best of the python2 and python3 current approaches: "Invalid character XX in line YY" and just reveal nothing about what lexical category — like identifier — python thinks the char is coming in.
The XX is like python2 and the YY like python3 If it can do better than '\xe2' — ie a codepoint — that’s a bonus but not strictly necessary
--
--- You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-ideas/-gsjDSht8VU/unsubscribe. To unsubscribe from this group and all its topics, send an email to python-ideas+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/d/optout. _______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
--
--- You received this message because you are subscribed to a topic in the Google Groups "python-ideas" group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/python-ideas/-gsjDSht8VU/unsubscribe. To unsubscribe from this group and all its topics, send an email to python-ideas+unsubscribe@googlegroups.com. For more options, visit https://groups.google.com/d/optout.