[Python-Dev] Security implications of pep 383

[Laura Creighton]

In a message of Tue, 29 Mar 2011 19:23:25 BST, Michael Foord writes:
>Hey all,
>
>Not sure how real the security risk is here:
>
>     http://blog.omega-prime.co.uk/?p=107
>
>Basically  he is saying that if you store a list of blacklisted files 
>with names encoded in big-5 (or some other non-utf8 compatible encoding) 
>if those names are passed at the command line, or otherwise read in and 
>decoded from an assumed-utf8 source with surrogate escaping, the 
>surrogate escape decoded names will not match the properly decoded 
>blacklisted names.

>All the best,
>
>Michael Foord
>

I am not sure there are any security related gotchas here.  All he is
saying is that if you decode the same bytestring using two different
encodings, you will get two different unicode strings (which therefore
will compare unequal).  Where's the problem, except in that you might
have unrealistic expectations?

Laura