On Sun, Nov 14, 2021 at 4:53 PM Steven D'Aprano <steve@pearwood.info> wrote:
Out of all the approximately thousand bazillion ways to write obfuscated
Python code, which may or may not be malicious, why are Unicode
confusables worth this level of angst and concern?

I for one am not full of angst nor particularly concerned. Though ti's a fine idea to inform folks about h this issues.

I am, however, surprised and disappointed by the NKFC normalization.

For example, in writing math we often use different scripts to mean different things (e.g. TeX's 
Blackboard Bold). So if I were to use some of the Unicode Mathematical Alphanumeric Symbols, I wouldn't want them to get normalized.

Then there's the question of when this normalization happens (and when it doesn't). If one is doing any kind of metaprogramming, even just using getattr() and setattr(), things could get very confusing:

In [55]: class Junk:
    ...:     𝗵e𝓵𝔩º = "hello"
    ...:

In [56]: setattr(Junk, "ᵖ𝖗𝐢𝘯𝓽", "print")

In [57]: dir(Junk)
Out[57]:
 '__weakref__',
<snip>
 'hello',
 'ᵖ𝖗𝐢𝘯𝓽']

In [58]: Junk.hello
Out[58]: 'hello'

In [59]: Junk.𝗵e𝓵𝔩º
Out[59]: 'hello'

In [60]: Junk.print
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-60-f2a7d3de5d06> in <module>
----> 1 Junk.print

AttributeError: type object 'Junk' has no attribute 'print'

In [61]: Junk.ᵖ𝖗𝐢𝘯𝓽
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-61-004f4c8b2f07> in <module>
----> 1 Junk.ᵖ𝖗𝐢𝘯𝓽

AttributeError: type object 'Junk' has no attribute 'print'

In [62]: getattr(Junk, "ᵖ𝖗𝐢𝘯𝓽")
Out[62]: 'print'

Would a proposal to switch the normalization to NFC only have any hope of being accepted?

and/or adding normaliztion to setattr() and maybe other places where names are set in code?

-CHB

--
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython