On 5/18/18 10:31 AM, Stephan Houben wrote:
NFKC normalization is restricted to identifiers. Keywords "must be spelled exactly as written here."
> It is guaranteed to work by PEP-3131: > https://www.python.org/dev/peps/pep-3131 https://www.python.org/dev/peps/pep-3131 > > "All identifiers are converted into the normal form NFKC while parsing; > comparison of identifiers is based on NFKC." > > NFKC normalization means spam must be considered the same identifier as > 𝐬𝐩𝐚𝐦 .
It's not the NFKC normalization that I'm questioning. Its the fact that it is done too late to catch the use of a keyword.
See above.
I would think that the rule that normalization is restricted to identifiers says that it needs to happen AFTER keyword identification (otherwise it would have applied to the keyword). To follow the rules and flag identifiers that normalize to keywords, either you need to normalize early and tag text that had been changed by normalization so keywords could flag errors (but late enough that you don't normalize inside strings and such), or you need to normalize late (as is done) but then add a check to see if the text became the same as a keyword. Seems a shame to go to extra work to flag as an error something that really could only have been done intentionally, removing a 'feature' to help with backwards compatibility. -- Richard Damon