[Distutils] PEP 517 - specifying build system in pyproject.toml

[Nick Coghlan]

On 26 May 2017 at 00:04, Thomas Kluyver <thomas at kluyver.me.uk> wrote:
> On Thu, May 25, 2017, at 02:27 PM, Paul Moore wrote:
>> I'd be concerned here that we risk making the frontend UI a lot more
>> complex for little actual benefit. I'd rather we stick with the
>> current model, where a backend just has some output to pass through to
>> the frontend. Let's get a solution that works for that before adding
>> extra complexity, or we'll never get the PEP signed off.
>
> I'm inclined to agree that we're overcomplicating things. But if we
> can't agree on which simple-but-imperfect option to take, maybe it's
> worth trying to work out something more complex.
>
> My proposed addition to the PEP so far says this:
>
> The build frontend may capture stdout and/or stderr from the backend. If
> the backend detects that an output stream is not a terminal/console
> (e.g. ``not sys.stdout.isatty()``), it SHOULD ensure that any output it
> writes to that stream is UTF-8 encoded. The build frontend MUST NOT fail
> if captured output is not valid UTF-8, but it MAY not preserve all the
> information in that case (e.g. it may decode using the *replace* error
> handler in Python). If the output stream is a terminal, the build
> backend is responsible for presenting its output accurately, as for any
> program running in a terminal.
>
> We could add a paragraph like this:
>
> The backend may do some operations, such as running subprocesses, which
> produce output in an unknown encoding. To handle such output, the build
> frontend MAY (?) create an empty directory, and set the environment
> variable PEP517_BUILD_LOGS to the path of this directory for the
> backend. If this environment variable is set, the backend MAY create any
> number of files inside this directory containing additional output. This
> is designed to allow the use of encoding detection tools on this output.
> If files are created in this directory, frontends SHOULD display its
> location in their output, and MAY display the contents of the files.

Seeing it like this pushes me from "Eh, maybe?" to "No, definitely not" :)

So that gets us to the point where we're agreeing that your suggested
addition to the PEP is basically right, with the only remaining
question being whether or not we're happy with the section that says
"it SHOULD ensure that any output it writes to that stream is UTF-8
encoded".

For a Python with locale coercion enabled, we're going to get that by
default, so such environments will comply without backend developers
doing anything in particular. Frontends may also decide to implement
their own PEP 538 style locale coercion for the backend build process
when they're running in a non-UTF-8 locale - specifying UTF-8 as a
SHOULD in the PEP gives them implied permission to do that.

So I don't think this is going to place any undue burden on backend
developers for *nix systems - frontends will probably want to
implement PEP 538 style locale coercion for LC_CTYPE to handle cases
where tools rely on the default stream encoding, but I think that's
fine.

That leaves Windows, and there I'm prepared to defer to Steve Dower's
opinion that it's better to deal with the encoding challenges of
consuming the output from MSVC in the build backend, rather than
expecting the frontend to deal with it. We also have a precedent now
in pip's legacy subprocess handling for what doing that reliably looks
like, so it shouldn't be hard for backend implementors to re-use that
approach as needed.

So I'm inclined to accept the encoding amendment, and then
provisionally accept the overall PEP pending implementation in pip.

I'll give others a couple of days to comment further, but assuming
nothing else comes up, I'll go ahead and do that on the weekend :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia