[core-workflow] codecov.io on PRs feedback

[Barry Warsaw]

On Feb 16, 2017, at 10:09 AM, Matthias Bussonnier wrote:

>Travis CI is doing coverage, and it pushes coverage.xml to codecov.io.
>Codecov will "just" aggregate coverage report from different travis-ci matrix
>entries to get the complete coverage. You can push coverage to codecov.io
>from your local machine, but that's painful.

Thanks!  This was the missing ingredient.  The .travis.hml encapsulates
exactly how the coverage integration works.  It probably wouldn't be too hard
to throw the appropriate before_script/script commands in a script or Makefile
target so folks could locally just do "make codecov" or some such (there's
already a "make coverage" but it doesn't do what you think it does).

>that's because Codecov is triggered and set the pending/failed/success status
>only once travis push the first coverage report. GitHub does not trigger
>codecov.  But I think you figured it out with your bottom comment.

Yep.  As it turns out, the Developers Guide does talk about coverage:

https://docs.python.org/devguide/coverage.html

but the .travis.yml gets around some of the problems by using a venv.

>That I'm usure. I would just strongly recommend codecov browser extension:
>https://github.com/codecov/support/wiki/Browser-Extension
>
>They add an overlay in the github UI that color the line numbers red/green
>depending on coverage. (PR diff and file browser) so you basically never
>have to go to the codecov website. [1]

That's fine.  What I really want is to be able to run coverage locally to see
what the results would be once I push my branch so I can fix anything obvious
beforehand.  I'm okay with going to codecov.io if there are any unexpected
coverage regressions.

One of the packages that I've grown very fond of is diff_cover.  It provides a
very nice way to see coverage of just your delta.  In my projects, I fail CI
if diff_cover reports < 100% even if the overall project isn't at 100% yet.
That way, I know all new code is covered, and I can implement a ratchet for
existing code.  codecov.io does something diff_cover like.

>>I would prefer to see a yellow dot (i.e. the test is still running).
>
>Agreed, I think we have to figure out how to have tests working with
>COVERAGE=True.  One possibility might be to push an empty coverage report on
>a fast branch. Docs ?

It would be worth experimenting with.

Thanks for the useful information!

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 801 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/core-workflow/attachments/20170216/925e3764/attachment.sig>