On Feb 13, 2017, at 9:49 AM, Victor Stinner victor.stinner@gmail.com wrote:
Hi,
I don't get the value of code coverage on a CI. I *like* running code coverage sometimes, but I don't like being annoying by "test failed: coverage -0,01%" by a change completely unrelated to code (note: this issue has been fixed be using a threshold of 1%). I'm annoyed by all these Codecov messages. Berker just told me that he even created a Gmail filter to drop all these email notifications...
What do you think of keeping this very useful feature, but not use it on pull requests: only run it on the branches (once changes are merged), to not "pollute" pull requests?
I think it’s incredibly useful on PRs, particularly because it can tell you if the PR has actually covered every line that it’s added or not. If you’re only periodically running coverage, then in my experience you generally miss covering a lot of lines accidentally. I’ve also never see the random -0.01% coverage of code in another project, and my guess is that there is some sort of non-determinism in the CPython test suite that would be a good idea to make deterministic anyways.
As with Alex, most projects I’ve been involved in turn off the comments and rely on the status checker.
— Donald Stufft