[New-bugs-announce] [issue16343] PyUnicode_FromFormatV() doesn't support utf-8 text

[Mariano Reingart]

New submission from Mariano Reingart:

Working in an internationalization proposal <http://python.org.ar/pyar/TracebackInternationalizationProposal>
I've stopped at #9769 where multi byte encodings (like utf-8) is not supported by PyUnicode_FromFormatV()

Beside my proposal, I think utf-8 should be supported for consistency with the other unicode functions, like PyUnicode_FromString() or even unicode_fromformat_arg()

Attached is a patch that:
- enhanced the iterator to detect multibyte sequences, with sanity checks about start & continuation bytes
- replaced unicode_write_cstr with PyUnicode_DecodeUTF8Stateful
- tests

Hope it helps, this is my first patch for cpython and my C skills are a bit rusty, so excuse me if there is any newbie glitch

----------
components: Interpreter Core, Unicode
files: pyunicode_fromformat_utf8.patch
keywords: patch
messages: 173996
nosy: ezio.melotti, reingart
priority: normal
severity: normal
status: open
title: PyUnicode_FromFormatV() doesn't support utf-8 text
type: enhancement
versions: Python 3.4
Added file: http://bugs.python.org/file27755/pyunicode_fromformat_utf8.patch

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue16343>
_______________________________________