Neil Schemenauer schrieb am 08.06.19 um 22:46:
It would be great if we had a system that did CI testing with the top PyPI modules. E.g. pull the latest versions of the top 100 PyPI modules and test them with the latest CPython branch.
FWIW, travis-ci provides the latest CPython master builds (and some latest dev branches). We use them in Cython for our CI tests.
One of the problems is that their images include some widely used libraries like NumPy, some of which in turn depend on Cython these days, so it happened once already that they failed to provide updated images because they were lacking a Cython version that worked with them, and we didn't notice that a change in Cython was needed because the CI builds were continuing to use an outdated CPython master version. :) Ah, circular dependencies… I think they fixed something about that, though. It wasn't a problem this time, at least.
We also have the "Cython testbed", which we (irregularly) use before releases to check that we didn't break more than was broken before the release.
It's pretty much what was asked for here, just for Cython, and it turns out to be a considerable amount of work to keep this from breaking arbitrarily for the included projects, even without changing something in Cython along the way.
Thus, personally, I would prefer a decentralised CI approach, where interested/important projects test themselves against CPython master (which many of them do already), and have them report back when they notice an undermotivated breakage. Some projects do that with Cython (and CPython) already, and that works quite well so far and seems the least work for everyone.