[Python-Dev] PEP 383: Non-decodable Bytes in System Character Interfaces

[Ronald Oussoren]

For what it's worth, the OSX API's seem to behave as follows:

* If you create a file with an non-UTF8 name on a HFS+ filesystem the  
system automaticly encodes the name.

That is,  open(chr(255), 'w') will silently create a file named '%FF'  
instead of the name you'd expect on a unix system.

* If you mount an NFS filesystem from a linux host and that directory  
contains a file named chr(255)

- unix-level tools will see a file with the expected name (just like  
on linux)
- Cocoa's NSFileManager returns u"?" as the filename, that is when the  
filename cannot be decoded using UTF-8 the name returned by the high- 
level API is mangled. This is regardless of the setting of LANG.
- I haven't found a way yet to access files whose names are not valid  
UTF-8 using the high-level Cocoa API's.

The latter two are interesting because Cocoa has a unicode filesystem  
API on top of a POSIX C-API, just like Python 3.x. I guess the choosen  
behaviour works out on OSX (where users are unlikely to run into this  
issue), but could be more problematic on other POSIX systems.

Ronald

On 28 Apr, 2009, at 14:03, Michael Foord wrote:

> Paul Moore wrote:
>> 2009/4/28 Antoine Pitrou <solipsis at pitrou.net>:
>>
>>> Paul Moore <p.f.moore <at> gmail.com> writes:
>>>
>>>> I've yet to hear anyone claim that they would have an actual  
>>>> problem
>>>> with a specific piece of code they have written.
>>>>
>>> Yep, that's the problem. Lots of theoretical problems noone has  
>>> ever encountered
>>> brought up against a PEP which resolves some actual problems  
>>> people encounter on
>>> a regular basis.
>>>
>>> For the record, I'm +1 on the PEP being accepted and implemented  
>>> as soon as
>>> possible (preferably before 3.1).
>>>
>>
>> In case it's not clear, I am also +1 on the PEP as it stands.
>>
>
> Me 2
>
> Michael
>> Paul.
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>>
>
>
> -- 
> http://www.ironpythoninaction.com/
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ronaldoussoren%40mac.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2224 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20090428/302315a3/attachment.bin>