Code Coverage for C and Python
Hi all, I wanted to get a wider feedback on adding a code coverage CI for C and Python. Prototype PR Link: https://github.com/numpy/numpy/pull/30911 *Why add this CI:* - diff-cover provides a way to see coverage only on the changed files, which is good for big PRs to understand the UT compared to the main branch. - With the new GH actions-upload (v7), we can get a clickable link to the report in a single file. For now the PR implements a MD file format as HTML file leads to ~250MB file *Why not to add this CI:* - Today code coverage and dead code can be calculated locally by following this: https://numpy.org/doc/stable/dev/index.html#test-coverage. So there is not much value in running this for every PR Please let us know your thoughts on having this new CI action. Thanks, Ganesh (https://github.com/ganesh-k13)
On Wed, Mar 11, 2026 at 6:20 AM Ganesh Kathiresan via NumPy-Discussion < numpy-discussion@python.org> wrote:
Hi all,
I wanted to get a wider feedback on adding a code coverage CI for C and Python. Prototype PR Link: https://github.com/numpy/numpy/pull/30911
*Why add this CI:* - diff-cover provides a way to see coverage only on the changed files, which is good for big PRs to understand the UT compared to the main branch. - With the new GH actions-upload (v7), we can get a clickable link to the report in a single file. For now the PR implements a MD file format as HTML file leads to ~250MB file
*Why not to add this CI:* - Today code coverage and dead code can be calculated locally by following this: https://numpy.org/doc/stable/dev/index.html#test-coverage. So there is not much value in running this for every PR
Please let us know your thoughts on having this new CI action.
Code coverage fixes might be a good use case for AI. I'd rather see a tool that could cover various things and be used to generate "problems" that need to be solved. There are currently two tools in the `tools` directory, I am not clear on how useful they are -- the C one uses valgrind -- but I'd suggest something like a spin tool to generate code coverage reports independently of CI. Chuck
There are currently two tools in the `tools` directory, I am not clear on how useful they are
This is outdated today (or rather there is an easier way) and needs to be replaced with `spin test --gcov`. This generated .gcda and .gcno files which are later used by gcovr to generate HTML reports. Thanks, Ganesh On Fri, Mar 13, 2026 at 11:50 PM Charles R Harris <charlesr.harris@gmail.com> wrote:
On Wed, Mar 11, 2026 at 6:20 AM Ganesh Kathiresan via NumPy-Discussion < numpy-discussion@python.org> wrote:
Hi all,
I wanted to get a wider feedback on adding a code coverage CI for C and Python. Prototype PR Link: https://github.com/numpy/numpy/pull/30911
*Why add this CI:* - diff-cover provides a way to see coverage only on the changed files, which is good for big PRs to understand the UT compared to the main branch. - With the new GH actions-upload (v7), we can get a clickable link to the report in a single file. For now the PR implements a MD file format as HTML file leads to ~250MB file
*Why not to add this CI:* - Today code coverage and dead code can be calculated locally by following this: https://numpy.org/doc/stable/dev/index.html#test-coverage. So there is not much value in running this for every PR
Please let us know your thoughts on having this new CI action.
Code coverage fixes might be a good use case for AI. I'd rather see a tool that could cover various things and be used to generate "problems" that need to be solved. There are currently two tools in the `tools` directory, I am not clear on how useful they are -- the C one uses valgrind -- but I'd suggest something like a spin tool to generate code coverage reports independently of CI.
Chuck
On Sun, Mar 15, 2026 at 9:48 PM Ganesh Kathiresan <ganesh3597@gmail.com> wrote:
There are currently two tools in the `tools` directory, I am not clear on
how useful they are
This is outdated today (or rather there is an easier way) and needs to be replaced with `spin test --gcov`. This generated .gcda and .gcno files which are later used by gcovr to generate HTML reports.
That fails for the *c.src files, seems that the preprocessing needs to be done. I'm wondering if the reports are something that can be digested by AI? <snip> Chuck
participants (2)
-
Charles R Harris -
Ganesh Kathiresan