[Distutils] command line versus python API for build system abstraction (was Re: build system abstraction PEP)

[Paul Moore]

On 11 November 2015 at 18:38, Nathaniel Smith <njs at pobox.com> wrote:
> Have you tried current dev versions of pip recently?

No, but I did see your work on this, and I appreciate and approve of it.

> The default now is to
> suppress the actual output but for progress reporting to show a spinner that
> rotates each time a line of text would have been printed. It's low tech but
> IMHO very effective. (And obviously you can also flip a switch to either see
> all or nothing of the output as well, or if that isn't there now if books
> really be added.) So I kinda feel like these are solved problems.

And this relies on build tools outputting to stdout, not stderr, and
not buffering their output.

That's an interface spec. Not everything has to be massively
complicated, and I wasn't implying it needed to be. Just that we need
conventions. One constant annoyance for pip is that distutils doesn't
properly separate stdout and stderr, so we can't suppress unnecessary
status reports without losing important error messages. Users report
this as a bug in pip, not in distutils, and I don't imagine that would
change if a project was using <name your build tool here>.

>
>> Taking all of those requirements into account, pip *has* to have some
>> level of control over the output of a build tool - with setuptools at
>> the moment, we have no such control (other than "we may or may not
>> show the output to the user") and that means we struggle to
>> realistically satisfy all of the conflicting requirements we have.
>>
>> So we do need much better defined contracts over stdin, stdout and
>> stderr, and return codes. This is true whether or not the build system
>> is invoked via a Python API or a CLI.
>
> Even if you really do want to define a generic structured system for build
> progress reporting (it feels pretty second-systemy to me), then in the
> python api approach there are better options than trying to define a
> specific protocol on stdout.

No, no, no. I never said that. All I was saying was that we need a
level of agreement on what pip can expect to do with stdout and
stderr, *given that there are known requirements pip's users expect to
be satisfied*.

Paul