[Distutils] PEP 517 - specifying build system in pyproject.toml
steve.dower at python.org
Sat May 20 19:42:44 EDT 2017
On 20May2017 1315, Paul Moore wrote:
> On 20 May 2017 at 17:36, Steve Dower <steve.dower at python.org> wrote:
>> In general, since most subprocesses (at least on Windows) do not have
>> customizable encodings, the tool that launches them needs to know what the
>> encoding is. Since we don't live in a Python 3.6 world quite yet, that means
>> the tool should read raw bytes from the compiler and encode them to UTF-8.
> Did you spot my point that Visual C produces output that's a mixture
> of OEM and ANSI codepages?
Yes, and it's a perfect example of why the MSVC-specific wrapper should
be the one to deal with tool encodings. If you forward unencoded bytes
like this back to the UI, it will have to deal with the mixed encoding.
> I'd be very surprised if build tool developers got this sort of edge
> case correct without at least some guidance from the PEP on the sorts
> of things they need to consider. You suggest "read raw bytes and
> encode them to UTF-8" - but you don't encode bytes, you encode
> strings, so you still need to convert those bytes to a string first,
> and there's no encoding you can reliably use for this. You need to use
> "errors=replace" to ensure you can handle inconsistently encoded data
> without getting an exception.
The "read raw bytes and [transcode] them" comment was meant to be that
sort of help. I didn't go as far as writing
errors="replace")`, but that's basically what I meant to imply. The
build tool developer is the *only* developer who can get this right, and
if they can't, then they have to figure out the most appropriate way to
work around the fact that they can't.
As for defining distutils as incompatible with the PEP, I'm okay with
that. Updating distutils to use subprocess for launching tools rather
than spawnv would be a very good start (and likely a good contribution
for a new contributor), but allowing build tools to continue to be
written badly is not worthwhile.
More information about the Distutils-SIG