[Distutils] PEP 517 - specifying build system in pyproject.toml

[Paul Moore]

On 19 May 2017 at 15:18, Thomas Kluyver <thomas at kluyver.me.uk> wrote:
> Hi,
>
> I'd like to make another push for PEP 517, which would make it possible
> to build wheels from a source tree with other build tools, without
> needing setup.py.

A point that came up recently while dealing with a pip issue.

"""
All hooks are run with working directory set to the root of the source
tree, and MAY print arbitrary informational text on stdout and stderr.
"""

Can we specify what encoding the informational text must be written
in? At the moment pip has problems dealing with non-ASCII locales
because it captures the build output and then displays it on error.
This involves a decode/encode step (on Python 3) or printing arbitrary
bytes to stdout (on Python 2). And at the moment we get UnicodeErrors
if there's a mismatch. I've patched it to use errors=replace, but we
still risk mojibake.

Ideally, we should specify an encoding that hooks will use for output
- but that's somewhat difficult as many build tools will want to do
things like run compilers which could do arbitrarily silly things. I
believe this is less of a problem on Unix (where there's a
well-managed convention), but on Windows there's an "OEM" codepage for
console programs, and an "ANSI" codepage for Windows programs - but
not all programs use the same one - some console programs such as
Mingw, I think, and Python itself if stdout is redirected (see
https://docs.python.org/3.6/library/sys.html#sys.stdout) use the ANSI
codepage.

So we may have to fudge the situation a bit. (Maybe something like
"Install tools MAY assume a specific encoding for the output, and MAY
produce corrupted output if the build tool does not use that encoding,
but install tools MUST NOT fail with an encoding error just because
the encodings don't match").

But I don't think we should leave the situation completely unspecified.

Paul.