[Python-Dev] File system path encoding on Windows

[Nick Coghlan]

On 31 August 2016 at 01:06, Victor Stinner <victor.stinner at gmail.com> wrote:
> 2016-08-30 16:31 GMT+02:00 Steve Dower <steve.dower at python.org>:
>> Any system that requires communication between two different versions of
>> Python must have install instructions (if it's public) or someone who
>> maintains it. It won't magically break without an upgrade, and it should not
>> get an upgrade without testing. The environment variable is available for
>> this kind of scenario, though I'd hope the testing occurs during beta and it
>> gets fixed by the time we release.
>
> I disagree that breaking backward compatibility is worth it. Most
> users don't care of Unicode since their application already "just
> works well" for their use case.
>
> Having to set an env var to "repair" their app to be able to upgrade
> Python is not really convenient.

This seems to be the crux of the disagreement: our perceptions of the
relative risks to native Windows Python applications that currently
work properly on Python 3.5 vs the potential compatibility benefits to
primarily *nix applications that currently *don't* work on Windows
under Python 3.5.

If I'm understanding Steve's position correctly, his view is that
native Python applications that are working well on Windows under
Python 3.5 *must already be using strings to interact with the OS*.
This means that they will be unaffected by the proposed changes, as
the proposed changes only impact attempts to pass bytes to the OS, not
attempts to pass strings.

In uncontrolled environments, using bytes to interact with the OS on
Windows just *plain doesn't work properly* under the current model, so
the proposed change is a matter of changing precisely how those
applications fail, rather than moving them from a previously working
state to a newly broken state.

For the proposed default behaviour change to cause problems then,
there must be large bodies of software that exist in sufficiently
controlled environments that they can get bytes-on-WIndows to work in
the first place by carefully managing the code page settings, but then
*also* permit uncontrolled upgrades to Python 3.6 without first
learning that they need to add a new environment variable setting to
preserve the Python 3.5 (and earlier) bytes handling behaviour.
Steve's assertion is that this intersection of "managed code page
settings" and "unmanaged Python upgrades" results in the null set.

A new opt-in config option eliminates any risk of breaking anything,
but means Steve will have to wait until 3.7 to try out the idea of
having more *nix centric software "just work" on Windows. In the grand
scheme of things, I still think it's worth taking that additional
time, especially if things are designed so that embedding applications
can easily flip the default behaviour.

Yes, there will be environments on Windows where the command line
option won't help, just as there are environments on Linux where it
won't help. I think that's OK, as we can use the 3.6 cycle to thrash
out the details of the new behaviour in the environments where it
*does* help (e.g. developers running their test suites on Windows
systems), get people used to the idea that the behaviour of binary
paths on Windows is going to change, and then actually make the switch
in Python 3.7.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia