[Distutils] Getting more momentum for pip

[Paul Moore]

On 5 March 2015 at 18:58, Donald Stufft <donald at stufft.io> wrote:
> Yet another issue is that pip’s test suite is not particularly very good.
> We’re missing a lot of coverage and we don’t have *any* CI running on
> platforms other than Ubuntu. This means that merging things is somewhat
> “dangerous” because it’s easy to break things without noticing unless you
> pull down the change and manually test things you can try. Even then that’s
> not good enough unless you can test it on other platforms as well. I’m sure
> Paul can fill in the blank on how often the test suite simply doesn’t run on
> Windows because of some POSIX assumption snuck in somewhere.

The test suite is pretty much broken on Windows, from what I recall. I
intended at one point to try to get it running cleanly on Windows, but
it was soul-destroyingly hard work, and I never got very far with it.
Given that there's no good Windows CI service (Appveyor is great but
it's very slow even on simple projects, and I think it has limits on
how long test suites can run so I'm not even sure we could run pip's
tests on it) I fear that any work done getting the test suite working
on Windows would pretty quickly regress...

If there was one thing on the infrastructure and support side of
things that would help enormously, it would be someone setting up CI
services for more platforms - Windows in particular, but things like
the ancient RHEL systems that people keep having issues on would also
be good. And resource willing to get the test suite working on those
platforms. (I'd be happy to help someone work on fixing the test suite
on Windows, but I really don't have the time to do it all myself).

> Other things that would help are:
>
> * People doing in-depth reviews of the current PRs that are there and
> suggesting changes or pointing out issues, etc.

Very much so. Anyone can add review comments to PRs. We could (and
probably should) document some quality guidelines for PRs (must
include a test, must include docs if there's a user-visible impact,
must be cross-platform, must work on Python 2 and 3). Having more
people who can test PRs on a wider range of platforms (Windows!!!)
would be great too - a simple comment "checked and confirmed on
Windows" is a great help.

> * People triaging issues (unfortunately this one isn’t super easy with
> GitHub Issues since you have to be a committer to change these things).

Hmm, that's a problem - but yes, even if they can only add comments,
saying "Please close as unreproducible", "Duplicate of XXX", "Please
add label YYY" would be helpful. The committers could trawl such
comments occasionally and action them.

> * People going through and reviewing old issues and PRs to try and figure
> out if the situations that caused them to be opened originally still apply
> or if that problem has been fixed or if the code has changed significantly
> enough that it’s likely to not longer exist.

Oh yes, please. And in particular, the old issues with repeated "+1"
or "me, too" comments, ping all the people who said "me too" and ask
them if they can provide a patch. And weeding out issues that only
apply when using ancient versions of setuptools, that sort of thing.

> These sorts of things would make it *much* easier to merge new things
> because there would be less risk and less things involved in actually going
> through and figuring out if any particular merge is a good idea or not. I
> also think that people willing to put in the work to do things like this
> would be good candidates for becoming core developers themselves, which
> would also help by increasing the number of people we have able to review
> and commit.

+1

Paul