Make AppVeyor CI non-mandatory during the CPython sprint?
Hi,
I see a high activity on pull requests on the CPython project, likely because of the CPython sprint. Sadly, the AppVeyor is still slow and still has only 2 jobs in parallel.
Would it be possible to make the AppVeyor job non-mandatory to allow to merge a PR faster? If it's configurable per branch, it would be nice to do this change for 2.7, 3.7, 3.8 and master branches.
I will keep an eye on buildbots to see if something bad happens :-)
Victor
On Mon, Sep 9, 2019 at 4:51 PM Victor Stinner vstinner@python.org wrote:
Hi,
I see a high activity on pull requests on the CPython project, likely because of the CPython sprint. Sadly, the AppVeyor is still slow and still has only 2 jobs in parallel.
Would it be possible to make the AppVeyor job non-mandatory to allow to merge a PR faster? If it's configurable per branch, it would be nice to do this change for 2.7, 3.7, 3.8 and master branches.
I'm in favor of this, though we should perhaps just switch from AppVeyor to Azure as the required check. Azure seems to have been stable enough, and is certainly faster than AppVeyor. I wouldn't much like to have no Windows build required in pre-merge CI.
On Mon, Sep 9, 2019, at 16:50, Victor Stinner wrote:
Hi,
I see a high activity on pull requests on the CPython project, likely because of the CPython sprint. Sadly, the AppVeyor is still slow and still has only 2 jobs in parallel.
Yeah, there has been quite some crankiness about this at the sprint. :)
Would it be possible to make the AppVeyor job non-mandatory to allow to merge a PR faster? If it's configurable per branch, it would be nice to do this change for 2.7, 3.7, 3.8 and master branches.
No one could think of a reason not to replace AppVeyor with Azure, so I've gone ahead and done that on all those branches.
I will keep an eye on buildbots to see if something bad happens :-)
What a hero!
On 9/9/2019 12:46 PM, Benjamin Peterson wrote:
On Mon, Sep 9, 2019, at 16:50, Victor Stinner wrote:
Hi,
I see a high activity on pull requests on the CPython project, likely because of the CPython sprint. Sadly, the AppVeyor is still slow and still has only 2 jobs in parallel.
A couple of years ago, I suggested that we treat testing time as a limited resource and not run unneeded tests. I still think this a good idea. The problem will only get worse in the future.
For instance, a PR properly labelled 'documentation' does not need code checks. I don't believe that a PR with only such changes needs any check other than to verify that changes in .py files are limited to docstrings and comments. Similarly for C comments.
Yeah, there has been quite some crankiness about this at the sprint. :)
Would it be possible to make the AppVeyor job non-mandatory to allow to merge a PR faster? If it's configurable per branch, it would be nice to do this change for 2.7, 3.7, 3.8 and master branches.
No one could think of a reason not to replace AppVeyor with Azure, so I've gone ahead and done that on all those branches.
Azure sometimes has false positive failures on the mac test where it times out after an hour because one of the tests hangs. This has happened to me twice in the last month. Can you make that not a requirement?
If the timeout only applies to the main suite run, and not to retests, the timeout could be shorter.
I will keep an eye on buildbots to see if something bad happens :-)
What a hero!
python-committers mailing list -- python-committers@python.org To unsubscribe send an email to python-committers-leave@python.org https://mail.python.org/mailman3/lists/python-committers.python.org/ Message archived at https://mail.python.org/archives/list/python-committers@python.org/message/A... Code of Conduct: https://www.python.org/psf/codeofconduct/
On Mon, Sep 9, 2019, at 20:15, Terry Reedy wrote:
On 9/9/2019 12:46 PM, Benjamin Peterson wrote:
On Mon, Sep 9, 2019, at 16:50, Victor Stinner wrote:
Hi,
I see a high activity on pull requests on the CPython project, likely because of the CPython sprint. Sadly, the AppVeyor is still slow and still has only 2 jobs in parallel.
A couple of years ago, I suggested that we treat testing time as a limited resource and not run unneeded tests. I still think this a good idea. The problem will only get worse in the future.
For instance, a PR properly labelled 'documentation' does not need code checks. I don't believe that a PR with only such changes needs any check other than to verify that changes in .py files are limited to docstrings and comments. Similarly for C comments.
We already prune the full test suite from documentation-only changes. (See "before_install" in .travis.yml.)
Figuring out how to detect whether a change is just comments is probably not worth it; there aren't really many PRs of that nature.
Yeah, there has been quite some crankiness about this at the sprint. :)
Would it be possible to make the AppVeyor job non-mandatory to allow to merge a PR faster? If it's configurable per branch, it would be nice to do this change for 2.7, 3.7, 3.8 and master branches.
No one could think of a reason not to replace AppVeyor with Azure, so I've gone ahead and done that on all those branches.
Azure sometimes has false positive failures on the mac test where it times out after an hour because one of the tests hangs. This has happened to me twice in the last month. Can you make that not a requirement?
It would be best if we could figure out how to fix that problem.
Le lun. 9 sept. 2019 à 18:46, Benjamin Peterson benjamin@python.org a écrit :
No one could think of a reason not to replace AppVeyor with Azure, so I've gone ahead and done that on all those branches.
Here is a reason to not replace AppVEyor with Azure: https://bugs.python.org/issue37245
The macOS job of Azure fails randomly, I reported the bug in June, no one managed to fix it yet (Steve just made progress on it, thanks!), and the whole Azure job is marked as "failed" if macOS fails. It seems like it's yet another multiprocessing bug.
Would it be possible to change the macOS job of Azure to mark it as optional?
Victor
On Tue, Sep 10, 2019, at 08:54, Victor Stinner wrote:
Le lun. 9 sept. 2019 à 18:46, Benjamin Peterson benjamin@python.org a écrit :
No one could think of a reason not to replace AppVeyor with Azure, so I've gone ahead and done that on all those branches.
Here is a reason to not replace AppVEyor with Azure: https://bugs.python.org/issue37245
The macOS job of Azure fails randomly, I reported the bug in June, no one managed to fix it yet (Steve just made progress on it, thanks!), and the whole Azure job is marked as "failed" if macOS fails. It seems like it's yet another multiprocessing bug.
Would it be possible to change the macOS job of Azure to mark it as optional?
Someone familiar with pipelines would have to answer that.
On 10Sep2019 0949, Benjamin Peterson wrote:
On Tue, Sep 10, 2019, at 08:54, Victor Stinner wrote:
Le lun. 9 sept. 2019 à 18:46, Benjamin Peterson benjamin@python.org a écrit :
No one could think of a reason not to replace AppVeyor with Azure, so I've gone ahead and done that on all those branches.
Here is a reason to not replace AppVEyor with Azure: https://bugs.python.org/issue37245
The macOS job of Azure fails randomly, I reported the bug in June, no one managed to fix it yet (Steve just made progress on it, thanks!), and the whole Azure job is marked as "failed" if macOS fails. It seems like it's yet another multiprocessing bug.
Would it be possible to change the macOS job of Azure to mark it as optional?
Someone familiar with pipelines would have to answer that.
I'm making a few updates to the Pipelines definition (which started as enabling coverage, and then disabling it because it's *soooo* slow and nobody seems to think we're using it on PRs anyway).
I believe I can make it so that the macOS test run times out after 20 minutes (~7 minutes longer than normal) and also that it's non-fatal, without making the build on that platform non-fatal. But I'm pretty convinced that the underlying issue here is a crash and we should diagnose it.
My work is at https://github.com/python/cpython/pull/15827 for anyone who'd like to follow along.
(Also, we don't have Pipelines configured for the 2.7 branch, so we should keep AppVeyor on there.)
Cheers, Steve
participants (5)
-
Benjamin Peterson -
Steve Dower -
Terry Reedy -
Victor Stinner -
Zachary Ware