[issue12263] punycode codec ignores the error handler argument

[STINNER Victor]

STINNER Victor <victor.stinner at haypocalc.com> added the comment:

Oh, I forgot to give a little bit more details.

b'abc\xff-'.decode('punycode', 'ignore') and b'abc\xff-'.decode('punycode', 'replace') raise a UnicodeDecodeError: the error handler is just useless (ignored) here.

With my patch, b'abc\xff-'.decode('punycode', 'ignore') gives 'abc'. (If I change the code to accept replace) b'abc\xff-'.decode('punycode', 'replace') gives also 'abc', but 'replace' doesn't work correctly in the part after "-" contain illegal byte sequences.

For example, b'a\xff-\xffb\xffga\xff'.decode("punycode", "replace") gives 'a�', whereas I would expect 'a�é' or 'aé�'. b'a-bga\xff'.decode("punycode", "replace") gives 'aé' as b'a-bga'.decode("punycode", "replace"), whereas I would expect 'aé�' or something like that.

> What's the point of disallowing the replace error handler?

It's just that I'm unable to patch punycode decoder to support the replace handler. Do you want to "implement" it?

> That's a slightly incompatible change, isn't it?

I don't think so because I consider that the punycode decoder never supported error handlers (other than strict) in Python 3.

What do you think?

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue12263>
_______________________________________