On Feb 16, 2017, at 10:09 AM, Matthias Bussonnier wrote:
Travis CI is doing coverage, and it pushes coverage.xml to codecov.io. Codecov will "just" aggregate coverage report from different travis-ci matrix entries to get the complete coverage. You can push coverage to codecov.io from your local machine, but that's painful.
Thanks! This was the missing ingredient. The .travis.hml encapsulates exactly how the coverage integration works. It probably wouldn't be too hard to throw the appropriate before_script/script commands in a script or Makefile target so folks could locally just do "make codecov" or some such (there's already a "make coverage" but it doesn't do what you think it does).
that's because Codecov is triggered and set the pending/failed/success status only once travis push the first coverage report. GitHub does not trigger codecov. But I think you figured it out with your bottom comment.
Yep. As it turns out, the Developers Guide does talk about coverage: https://docs.python.org/devguide/coverage.html but the .travis.yml gets around some of the problems by using a venv.
That I'm usure. I would just strongly recommend codecov browser extension: https://github.com/codecov/support/wiki/Browser-Extension
They add an overlay in the github UI that color the line numbers red/green depending on coverage. (PR diff and file browser) so you basically never have to go to the codecov website. [1]
That's fine. What I really want is to be able to run coverage locally to see what the results would be once I push my branch so I can fix anything obvious beforehand. I'm okay with going to codecov.io if there are any unexpected coverage regressions. One of the packages that I've grown very fond of is diff_cover. It provides a very nice way to see coverage of just your delta. In my projects, I fail CI if diff_cover reports < 100% even if the overall project isn't at 100% yet. That way, I know all new code is covered, and I can implement a ratchet for existing code. codecov.io does something diff_cover like.
I would prefer to see a yellow dot (i.e. the test is still running).
Agreed, I think we have to figure out how to have tests working with COVERAGE=True. One possibility might be to push an empty coverage report on a fast branch. Docs ?
It would be worth experimenting with. Thanks for the useful information! Cheers, -Barry