[Python-Dev] Python-3.0, unicode, and os.environ

[Michael Urman]

On Fri, Dec 5, 2008 at 18:48, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Toshio Kuratomi wrote:
>> Nick Coghlan wrote:
>>> Toshio Kuratomi wrote:
>>>> Guido van Rossum wrote:
>>>>> Glob was just an example. Many use cases for directory traversal
>>>>> couldn't care less if they see *all* files.
>>>>>
>>>> Okay.  Makes it harder to prove correct or not if I don't know what the
>>>> use case is :-)  I can't think of a single use case off-hand.
>>>>
>>>> Even your example of a ??.txt file making retrieval of *.py files fail
>>>> is a little broken.  If there was a ??.py file that was undecodable the
>>>> program would most likely want to know that file existed.
>>> Why? Most programs won't be able to do anything with it. And if the
>>> program *can* do something with it... that's what the bytes version of
>>> the APIs are for.
>>>
>> Nonsense.  A program can do tons of things with a non-decodable
>> filename.  Where it's limited is non-decodable filedata.
>
> You can't display a non-decodable filename to the user, hence the user
> will have no idea what they're working on. Non-filesystem related apps
> have no business trying to deal with insane filenames.

And what of python's batteries---does a library that takes filenames
or directories from a controlling program and processes the contents
of the file need to care whether the file can be encoded properly? Is
said library filesystem related or not?

Won't it be awful when it's the directory name, and processing the
file works if you change into its directory, but not if you're outside
of it? And if there's an error during processing and the library
reports a full filename using os.abspath("file.ext"), but cannot get
the results?

> Linux is moving towards a standard of UTF-8 for filenames, and once we
> get to the point where the idea of encoding filenames and environment
> variables any other way is seen as crazy, then the Python 3 approach
> will work seamlessly.
>
> In the meantime, raw bytes APIs will provide an alternative for those
> that disagree with that philosophy.

And until that time, it's agony for the library writers who didn't
think they needed to care, but find that their users (other
developers) do.
-- 
Michael Urman