Add support for generating coverage reports w/ gcov - #4489
Open
d-torrance wants to merge 9 commits into
Open
Conversation
Adds a --enable-gcov option (modeled on --enable-profile) that builds with
--coverage -O0, plus -fprofile-abs-path under gcc so gcov/gcovr can resolve
sources from the out-of-tree build; it also disables stripping.
Coverage data (.gcda) is generated automatically whenever any coverage-built
binary exits, so no target runs the tests for you: just run the unit tests,
make check, or the M2 binary on your own code. Two top-level targets clear
and report on the accumulated data:
coverage-reset delete accumulated .gcda (tree-wide)
coverage-report produce an HTML report via gcovr (engine-scoped by default;
override COVERAGE_ROOT/COVERAGE_OBJDIR for the whole tree)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Adds a -DGCOV=ON option that builds with --coverage -O0, plus -fprofile-abs-path under gcc, mirroring the PROFILING flag block. New cmake/gcov.cmake provides coverage-reset and coverage-report (gcovr) targets, paralleling the autotools setup; coverage data is generated by running any coverage-built binary (unit tests, ctest, or the M2 binary). Also fixes a pre-existing typo in the dormant Clang coverage.cmake, which searched for clang-format instead of llvm-cov. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The coverage-report target now prints the report path as an OSC 8 terminal hyperlink (file://host/path) via printf, so terminals that support it render it as a clickable link. Done for both the autotools (Makefile.in) and cmake (gcov.cmake) targets. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gcov can report the same function on more than one line (e.g. inlines compiled at -O0), which gcovr's default --merge-mode-functions=strict treats as a fatal error. Pass merge-use-line-min so those entries are merged and attributed to the function's first line. Applied to both the autotools and cmake targets; the autotools value is overridable via COVERAGE_MERGE_MODE. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
-fprofile-abs-path is gcc-only (clang rejects it), and it was only a convenience for source resolution -- which gcovr already handles via its --root/--object-directory options. Removing it keeps --coverage portable across gcc and clang and avoids a compiler-specific guard. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Consolidate the declaration and flag-setting logic into one block, quote macro arguments with [...], use AS_IF instead of raw test/if, and keep lines within 80 columns. No functional change. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
On macOS, applying --coverage before AC_FC_LIBRARY_LDFLAGS/AX_BLAS/AX_LAPACK let gfortran expand --coverage to -lgcov and bake it into FCLIBS/BLAS_LIBS. The final clang link then pulled in both gcc's libgcov and clang's own libclang_rt.profile, producing duplicate __gcov_reset/__gcov_fork symbols. Move the gcov flag block after that detection so only the compiler's own coverage runtime ends up on the link line. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Pass the object tree to gcovr as a positional search path instead of via --object-directory. --object-directory forces gcov to run in a single directory, which cannot resolve the relative source paths of objects built in deeper VPATH subdirectories (e.g. Macaulay2/e/unit-tests), causing "could not infer a working directory" errors. As a search path, gcovr runs gcov in each data file's own directory, so every source resolves. Also default COVERAGE_ROOT/COVERAGE_OBJDIR to the whole tree, drop the hardcoded unit-tests exclude, and add a GCOVR_OPTIONS variable for passing extra gcovr options (filters, excludes, formats). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Report over the whole tree by default (root = source tree, build tree passed as a positional search path rather than --object-directory), drop the hardcoded unit-tests exclude, and add a GCOVR_OPTIONS cache variable for passing extra gcovr options (split with separate_arguments). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Member
|
I suppose we need to install gcovr first? Where should we put that info? I don't want (or can't) install using pip globally... |
Member
|
Here is one set of instructions (from chatgpt). Sound reasonable? (I've never used pipx before). Do the following once (using homebrew, on mac) Restart a terminal, do to upgrade or remove: |
Member
Author
|
It's available from most package managers ( |
Member
Author
|
Yeah |
Member
|
The info about pipx suggested by chatGPT is too complicated and not needed on a mac: I believe just doing |
|
|
||
| if(GCOV) | ||
| find_program(GCOVR NAMES gcovr) | ||
| set(_coverage_dir ${CMAKE_BINARY_DIR}/coverage) |
Contributor
There was a problem hiding this comment.
Could we add a small CI smoke test with coverage enabled, especially for the CMake path? The current checks all use the default GCOV=OFF, so they don't exercise the instrumentation flags or either coverage target; the PR also notes that the CMake support hasn't been tested. Even a minimal configure/build/run/coverage-report job would catch toolchain and gcovr invocation problems.
AI-assisted review comment made by GPT 4.6-Sol under Andrew's direction in order to observe deeper code reaches.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This replaces MichaelABurr#65
As discussed in the C++ refactoring group Zoom meeting yesterday: If we build M2 using
./configure --enable-gcov, then we can runmake coverage-reportto get a gcovr-generated coverage report of the engine and interpreter. In particular, everything that was covered since we built M2 (or ranmake coverage-reset).This will be useful to figuring out which features still need unit testing (e.g., by running
make coverage-reset && make -C Macaulay2/e check && make coverage-report, we can see what all the unit tests hit.) There's also CMake support that I haven't tested.AI Disclosure
🤖 This was Claude through and through 🤖