os.lisdir, gets unicode, returns unicode... USUALLY?!?!?

[Leo Kislov]

gabor wrote:
> Martin v. Löwis wrote:
> > gabor schrieb:
> >> i also recommend this approach.
> >>
> >> also, raising an exception goes well with the principle of the least
> >> surprise imho.
> >
> > Are you saying you wouldn't have been surprised if that had been
> > the behavior?
>
>
> yes, i would not have been surprised. because it's kind-of expected when
> dealing with input, that malformed input raises an unicode-exception.
> and i would also expect, that if os.listdir completed without raising an
> exception, then the returned data is correct.

The problem is that most programmers just don't want to deal with
filesystem garbage but they won't be happy if the program breaks
either.

> > How would you deal with that exception in your code?
>
> depends on the application. in the one where it happened i would just
> display an error message, and tell the admins to check the
> filesystem-encoding.
>
> (in other ones, where it's not critical to get the correct name, i would
> probably just convert the text to unicode using the "replace" behavior)
>
> what about using flags similar to how unicode() works? strict, ignore,
> replace and maybe keep-as-bytestring.
>
> like:
> os.listdir(dirname,'strict')

That's actually an interesting idea. The error handling modes could be:
'mix' -- current behaviour, 'ignore' -- drop names that cannot be
decoded, 'separate' -- see my other message. 

  -- Leo