Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore: codecov reports to match go coverage reports #8696

Merged
merged 1 commit into from
Jan 18, 2024

Conversation

amandavialva01
Copy link
Contributor

@amandavialva01 amandavialva01 commented Jan 16, 2024

Description

Currently, Codecov coverage reports do not precisely match the coverage reports generated by Go Codecov coverage tends to be lower than Go coverage output due to Codecov's defaults to ignore LOC that are not directly within the body of a function as well as marking certain LOC as partially covered or uncovered for branches that evaluate to false.

This PR overwrites those defaults to bring Codecov reports inline with Go coverage reports by

  • Counting all LOC in a file (as the Go cover tool does) when calculating coverage metrics and generating reports.
  • Changing lines marked by Codecov as partial to hit (so that Codecov coverage percentages are calculated to mark the code in such lines as "covered")

This PR also lowers the per-PR coverage target by 5% for Go code in the event that backend coverage status checks are required to pass before merging into main (to avoid too many blockers for PRs opened before this one).

Test Plan

NOTE: The test plan can be used on this commit.

  • Go to CircleCI runs and select the test-go status check.
  • Select go-coverage (CI run needs to be a success for all jobs in this workflow)
  • In the CircleCI dashboard for the go-coverage job, select "Artifacts" -> "go-coverage/master-coverage.html"
  • Select a random file from the drop-down (preferable a file with > 50 LOC and coverage > 40%. master/internal/api_experiments.go, for example, is a good file to choose).
    • Then, note the parenthesized coverage percentage next to the file path.
  • Now, return back to the Github UI where the CI builds are running and click on "Details" to the right of the codecov/patch/backend status check to go to the Codecov UI.
    • In the UI, under the given commit for which you checked the go-coverage artifacts, navigate to the "File explorer" tab and then navigate into the same file that you observed metrics for above.
    • Verify that Codecov coverage percentage (at the top right of the file) shows a percentage that is either
    • +/- 1% of the value you saw earlier (from the Go coverage artifact)
    • 1% higher than the Go coverage percentage.

Checklist

  • Changes have been manually QA'd
  • User-facing API changes need the "User-facing API Change" label.
  • Release notes should be added as a separate file under docs/release-notes/.
    See Release Note for details.
  • Licenses should be included for new code which was copied and/or modified from any external code.

Copy link

netlify bot commented Jan 16, 2024

Deploy Preview for determined-ui ready!

Name Link
🔨 Latest commit bd4731b
🔍 Latest deploy log https://app.netlify.com/sites/determined-ui/deploys/65a99017b4b268000880b763
😎 Deploy Preview https://deploy-preview-8696--determined-ui.netlify.app/
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@amandavialva01 amandavialva01 force-pushed the amanda/adjustBackendCoverageTarget branch from 29b07e6 to 9dcda1d Compare January 16, 2024 20:26
Copy link

codecov bot commented Jan 16, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (0782c35) 51.31% compared to head (bd4731b) 53.54%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8696      +/-   ##
==========================================
+ Coverage   51.31%   53.54%   +2.23%     
==========================================
  Files         894      894              
  Lines      156712   146985    -9727     
  Branches     2096     2094       -2     
==========================================
- Hits        80421    78710    -1711     
+ Misses      74795    68265    -6530     
+ Partials     1496       10    -1486     
Flag Coverage Δ
backend 40.87% <ø> (+5.16%) ⬆️
harness 64.29% <ø> (ø)
web 53.80% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

see 324 files with indirect coverage changes

@amandavialva01 amandavialva01 self-assigned this Jan 16, 2024
@amandavialva01 amandavialva01 force-pushed the amanda/adjustBackendCoverageTarget branch 2 times, most recently from 7c1bc1a to a09cb9d Compare January 16, 2024 22:01
@amandavialva01 amandavialva01 force-pushed the amanda/adjustBackendCoverageTarget branch from a09cb9d to f601dab Compare January 16, 2024 22:05
@amandavialva01 amandavialva01 force-pushed the amanda/adjustBackendCoverageTarget branch from f601dab to bd4731b Compare January 18, 2024 20:54
@amandavialva01 amandavialva01 enabled auto-merge (squash) January 18, 2024 21:08
@amandavialva01 amandavialva01 merged commit faea553 into main Jan 18, 2024
69 of 83 checks passed
@amandavialva01 amandavialva01 deleted the amanda/adjustBackendCoverageTarget branch January 18, 2024 21:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants