Since both Paul Moore and Antoine Pitrou started to ask questions about a
side comment I made about VSTS, I might as well start a discussion (Steve
has also *just* emailed python-committers about this topic).
Basically Steve Dower and I have been working directly with the VSTS team
to improve their Python support. Along the way they have announced they are
adding anonymous access to build logs which was always a major blocker for
using it in open source. VSTS gave us early access to the anonymous access
that they are launching along with 20 concurrent builders which is more
than what we currently have on Travis and AppVeyor.
At this point Steve has added the appropriate YAML files in the .vsts
directory that control the build jobs so we can test out the integration
and see what the performance is like. But based on what we have been seeing
there will be no queue in getting PRs tested and that applies to Windows,
macOS, and Linux (IOW better coverage and no more issues about performance
or having to wait too long for CI to go green ;) . There is also other
benefits like making at least Windows builds easier to release as signing
those builds can now be automated.
My expectation is that once we are convinced that the VSTS builds are
passing consistently and everything is working as everyone wants for e.g. a
week, we can plan to turn off Travis and AppVeyor for master and 3.x
branches (no one wants to bother with trying to get 2.7 to work on this and
we can simply continue to rely on Travis and AppVeyor as necessary for the
next 18 months). I don't think we want to run multiple CIs indefinitely on
the same branch as it does take time and effort to keep CI green on a
provider since they all have their own quirks.
I should also mention that this free access is not limited to just CPython
and any project can get free accounts and credits now, you just have to
talk to Steve about getting in on the anonymous access before it launches
to the general public (I have no timeline on when that's happening).
