[Distutils] Getting more momentum for pip

[Donald Stufft]

> On Mar 5, 2015, at 2:34 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 
> On 5 March 2015 at 18:58, Donald Stufft <donald at stufft.io> wrote:
>> Yet another issue is that pip’s test suite is not particularly very good.
>> We’re missing a lot of coverage and we don’t have *any* CI running on
>> platforms other than Ubuntu. This means that merging things is somewhat
>> “dangerous” because it’s easy to break things without noticing unless you
>> pull down the change and manually test things you can try. Even then that’s
>> not good enough unless you can test it on other platforms as well. I’m sure
>> Paul can fill in the blank on how often the test suite simply doesn’t run on
>> Windows because of some POSIX assumption snuck in somewhere.
> 
> The test suite is pretty much broken on Windows, from what I recall. I
> intended at one point to try to get it running cleanly on Windows, but
> it was soul-destroyingly hard work, and I never got very far with it.
> Given that there's no good Windows CI service (Appveyor is great but
> it's very slow even on simple projects, and I think it has limits on
> how long test suites can run so I'm not even sure we could run pip's
> tests on it) I fear that any work done getting the test suite working
> on Windows would pretty quickly regress...
> 
> If there was one thing on the infrastructure and support side of
> things that would help enormously, it would be someone setting up CI
> services for more platforms - Windows in particular, but things like
> the ancient RHEL systems that people keep having issues on would also
> be good. And resource willing to get the test suite working on those
> platforms. (I'd be happy to help someone work on fixing the test suite
> on Windows, but I really don't have the time to do it all myself).

Yea, I wouldn’t personally put effort into fixing the test suite on Windows
without something in place to ensure it doesn’t break again.

I know the folks behind Travis CI. I know they were looking at adding Windows
support and said that if someone can write a go app that boots up a Windows
Azure instance and then uses winrm to run a command on it, even echo, that
would get them a big step of the way towards being able to support it.

OSX and lots of other POSIX based systems are also not covered of course,
it would be great to setup something that other people can contribute
build machines to. We can spin up anything on a Rackspace cloud (and probably
other clouds too) but some things people might care about (AIX?) we can’t do.
I think it wouldn’t be unreasonable to say that for things we can’t run a
builder ourselves for, that people who care about that platform needs to
provide us with a suitable instance.

None of that matters much though without something that allows us to run tests
on more platforms than whatever Travis provides us. Ideally this would support
PR based testing (which means it needs some sort of VM or isolation support to
do it securely) but if some platforms can’t be easily virtualized like that then
a post merge trigger is acceptable too.

> 
>> Other things that would help are:
>> 
>> * People doing in-depth reviews of the current PRs that are there and
>> suggesting changes or pointing out issues, etc.
> 
> Very much so. Anyone can add review comments to PRs. We could (and
> probably should) document some quality guidelines for PRs (must
> include a test, must include docs if there's a user-visible impact,
> must be cross-platform, must work on Python 2 and 3). Having more
> people who can test PRs on a wider range of platforms (Windows!!!)
> would be great too - a simple comment "checked and confirmed on
> Windows" is a great help.

Yea, I don’t have a Windows machine so I’m often times just guessing if
it works on Windows, or pinging you for it. For major things I can spin
up a Windows VM but that takes a good 30+ minutes to do which again goes
back to that our current setup has a lot of time wasters for pip core.

> 
>> * People triaging issues (unfortunately this one isn’t super easy with
>> GitHub Issues since you have to be a committer to change these things).
> 
> Hmm, that's a problem - but yes, even if they can only add comments,
> saying "Please close as unreproducible", "Duplicate of XXX", "Please
> add label YYY" would be helpful. The committers could trawl such
> comments occasionally and action them.

I personally get emails for every issue, closing duplicates or adding
labels and such is something that takes 15 seconds to do if someone leaves
a comment like that. Another option is to move our issue tracker off of
Github into something else that supports non-committers being able to
manage the issue tracker. Empowering non core to do more things is another
thing that would be useful and requires someone to take the time to figure
out how we can best do that (switch away from GH issues? To What?) and then
actually do the work to make it happen (create salt states to deploy, create
scripts to migrate etc).

> 
>> * People going through and reviewing old issues and PRs to try and figure
>> out if the situations that caused them to be opened originally still apply
>> or if that problem has been fixed or if the code has changed significantly
>> enough that it’s likely to not longer exist.
> 
> Oh yes, please. And in particular, the old issues with repeated "+1"
> or "me, too" comments, ping all the people who said "me too" and ask
> them if they can provide a patch. And weeding out issues that only
> apply when using ancient versions of setuptools, that sort of thing.

I try to do this periodically but I mostly only do the ones that I can tell from
a glance that are safe to close.

> 
>> These sorts of things would make it *much* easier to merge new things
>> because there would be less risk and less things involved in actually going
>> through and figuring out if any particular merge is a good idea or not. I
>> also think that people willing to put in the work to do things like this
>> would be good candidates for becoming core developers themselves, which
>> would also help by increasing the number of people we have able to review
>> and commit.
> 
> +1
> 
> Paul

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 801 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/distutils-sig/attachments/20150305/cd0c0356/attachment-0001.sig>