[New-bugs-announce] [issue10209] Mac OS X: Decompose filenames on encode, and precompose filenames on decode

[STINNER Victor]

New submission from STINNER Victor <victor.stinner at haypocalc.com>:

PyUnicode_EncodeFSDefault() and os.fsencode() should decompose the filename (NFD) before encoding it to utf-8.

PyUnicode_DecodeFSDefault(AndSize)() and os.fsdecode() should precompose the filename (NFC) after decoding it from utf-8.

Qt library does this on Mac: see locale_encode()/locale_decode() (filename encoder/decoder) functions in src/corelib/io/qfile.cpp.

It should fix some issues of test_pep277 on Mac OS X (see #8423).

I'm not completly sure that we should do that :-)

(I used the nosy list from issues #4388 and #8423).

--

Technical Q&A QA1173, Text Encodings in VFS:
http://developer.apple.com/mac/library/qa/qa2001/qa1173.html

Q: I'm writing a file system (VFS) plug-in for Mac OS X. How do I handle text encodings correctly?
A: In Mac OS X's VFS API file names are, by definition, canonically decomposed Unicode, encoded using UTF-8. This raises a number of interesting issues. (...)

----------
assignee: ronaldoussoren
components: Interpreter Core, Macintosh, Unicode
messages: 119662
nosy: MrJean1, amaury.forgeotdarc, db3l, flox, haypo, ixokai, loewis, mark.dickinson, michael.foord, ned.deily, piro, pitrou, ronaldoussoren, rpetrov, skip.montanaro, slmnhq
priority: normal
severity: normal
status: open
title: Mac OS X: Decompose filenames on encode, and precompose filenames on decode
versions: Python 3.2

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue10209>
_______________________________________