[Python-Dev] File system path encoding on Windows

[Steve Dower]

As I've said before, on Windows this is a compatibility hack to make code written for other platforms work. Making the user opt in is not fair, and does not help improve the libraries that need it, because few people will change their library to work with a non default option.

The "developer" I'm concerned about doesn't need to turn this on - bytes work just about fine on POSIX (if you don't inspect the contents). It's the random user on Windows who installed their library that has the problem. They don't know the fix, and may not know how to apply it (e.g. if it's their Jupyter notebook that won't find one of their files - no obvious command line options here).

Any system that requires communication between two different versions of Python must have install instructions (if it's public) or someone who maintains it. It won't magically break without an upgrade, and it should not get an upgrade without testing. The environment variable is available for this kind of scenario, though I'd hope the testing occurs during beta and it gets fixed by the time we release.

Changing the locale encoding is something I'm quite happy to back out of. It's already easy enough for the developer to specify the encoding when opening a file, or to wrap open() and change their own default. But developers cannot change the encoding used by the os module, which is why we need to do it.

Cheers,
Steve

Top-posted from my Windows Phone

-----Original Message-----
From: "Victor Stinner" <victor.stinner at gmail.com>
Sent: ‎8/‎30/‎2016 1:21
To: "Nick Coghlan" <ncoghlan at gmail.com>
Cc: "Steve Dower" <steve.dower at python.org>; "Python Dev" <python-dev at python.org>
Subject: Re: [Python-Dev] File system path encoding on Windows

Le 30 août 2016 03:10, "Nick Coghlan" <ncoghlan at gmail.com> a écrit :
> However, this view is also why I don't agree with being aggressive in
> making this behaviour the default on Windows - I think we should make
> it readily available as a provisional feature through a single
> cross-platform command line switch and environment setting (e.g. "-X
> utf8" and "PYTHONASSUMEUTF8") so folks that need it can readily opt in
> to it,
I'm sorry, but I should have start from this point.
Modifyng the default or adding an option are completely different. I like the idea of adding a -X utf8 option on Windows. If it's an opt-in option, the developer takes the responsabilty of any possible backward incompatible change and plays the Unicode dance when handling input/output data with other applications.
My long email tries to explain why I consider that modifying the default in 3.6 is a strong NO for me.
> but we can defer making it the default until 3.7 after folks
> have had a full release cycle's worth of experience with it in the
> wild.
If Steve changes its project to add an option but don't change the default, I will help to make it happen before 3.6 and help to implement the UNIX part. It would even make it stronger if the option is "portable", even if the exact semantic is differnent between UNIX and Windows.
If the default doesn't change, I don't think that a PEP is required.
Later when we will get enough feedback, we will be able to decide to drop the option (if it was a very bad idea because of very bad feedback), or even make it the default on a platform (Windows).
Victor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20160830/864dca85/attachment-0001.html>