Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pytest ( versions 7.2+ ) with pytest-cov 4.0.0 fails to write xml report durring GH actions #49

Open
reactive-firewall opened this issue Jan 16, 2023 · 3 comments
Assignees
Labels
CI Continuous Integration Tooling GitHub Anything Meta Testing Something can be verified

Comments

@reactive-firewall
Copy link
Owner

Describe the bug
A clear and concise description of what the bug is.

To Reproduce
Steps to reproduce the behavior:

  1. ensure working version locally
  2. run the make test-pytest target
  3. ensure coverage output file is created
  4. compare to GH actions on any os and any python 3.7-3.10 (3.11 is an expected fail)
  5. See error

Expected behavior
CI should produce the same or similar behavior to running the same steps locally.

Logs (optional)
CI builds #138-#150

Screenshots (optional)
If applicable, add screenshots to help explain your problem.

  • Python Version [3.7-3.11]
  • Multicast Python Library Version: 1.4.2

Additional context (optional)
This seems to be related to only to coverage within pytest at the moment.

@reactive-firewall reactive-firewall self-assigned this Jan 16, 2023
@reactive-firewall reactive-firewall added this to To do in Multicast Project via automation Jan 16, 2023
@reactive-firewall reactive-firewall added Testing Something can be verified CI Continuous Integration Tooling GitHub Anything Meta labels Jan 16, 2023
@reactive-firewall
Copy link
Owner Author

Work around for now is to use the M.A.T.s test via make test and use coverage out-of-band to check test coverage.

@reactive-firewall
Copy link
Owner Author

🤔 Napkin code for a regression test of this:

#! /bin/bash
# ---> will need header and boilerplate code here <---
make test-pytest ; wait ;
test -r ./test-reports/junit.xml || EXIT_CODE=1 ;
exit $EXIT_CODE ;

reactive-firewall added a commit that referenced this issue Aug 23, 2024
…CI runs (- WIP #49 -)

Changes in file tests/check_codecov_regression:
 Unknown Changes
reactive-firewall added a commit that referenced this issue Aug 23, 2024
…-)

Changes in file .coveragerc:
 partial_branches =

Changes in file .github/workflows/Tests.yml:
 jobs:
reactive-firewall added a commit that referenced this issue Aug 23, 2024
* feature-PEP-517:
  [UPDATE] Version 1.4.3-RC
  [PEP-517] move to be PEP-517 compatable and away from setup.py (- WIP #46 -)

* feature:
  [TESTS] Fix for #49 by adding new test-unit for regressions (- WIP #49 -)
  [TESTS] added regression check for issue with test-reports in some CI runs (- WIP #49 -)

Changes in file .coveragerc:
 + hardened erronious coverage

Changes in file .github/workflows/Tests.yml:
 + new job: REGRESSION

Changes in file Makefile:
 + W.I.P. to move to new build system, see #46
 + improved stability a little

Changes in file multicast/__init__.py:
 + Version 1.4.3 bump

Changes in file requirements.txt:
 + added `build>=1.2.1` as part of #46

Changes in file setup.cfg:
 + Version 1.4.3 bump

Changes in file setup.py:
 - deprecated old build system

Changes in file tests/check_codecov_regression:
 + added new check to verify test-reports from `make test-pytest`
@reactive-firewall
Copy link
Owner Author

✅ Issue seems fixed as of pytest-cov v5.0+

reactive-firewall added a commit that referenced this issue Aug 27, 2024
#46 & #49 -)

Changes in file .github/workflows/Tests.yml:
 - use same setup as real test when checking for regressions

Changes in file tests/check_cc_lines:
 - workaround for missing `shlock`

Changes in file tests/check_codecov:
 - workaround for missing `shlock`
reactive-firewall added a commit that referenced this issue Aug 27, 2024
#46 & #49 -)

Changes in file .github/workflows/Tests.yml:
 - output more on regression testing
reactive-firewall added a commit that referenced this issue Aug 27, 2024
Changes in file .github/workflows/Tests.yml:
 FIXED TYPO
reactive-firewall added a commit that referenced this issue Aug 27, 2024
Changes in file .github/workflows/Tests.yml:
 minor weird tweaks
reactive-firewall added a commit that referenced this issue Aug 27, 2024
### ChangeLog:

Changes in file .github/workflows/Tests.yml:
 jobs:
reactive-firewall added a commit that referenced this issue Aug 27, 2024
### ChangeLog:

Changes in file Makefile:
 test-style: cleanup must_have_flake
reactive-firewall added a commit that referenced this issue Aug 28, 2024
… (- WIP #46 & #49 -)

### ChangeLog:

Changes in file tests/check_codecov_regression:
 elif [[ ( -d $(git rev-parse --show-toplevel 2>/dev/null) ) ]] ; then
reactive-firewall added a commit that referenced this issue Aug 28, 2024
… as it only yields false positives at the moment (- WIP #46 & #49 -)

### ChangeLog:

Changes in file .github/workflows/Tests.yml:
 jobs:
reactive-firewall added a commit that referenced this issue Aug 28, 2024
Changes in file .codecov.yml:
 removeed `setup.py` from coverage quota

Changes in file .coveragerc:
 removeed `setup.py` from coverage quota

Changes in file .github/labeler.yml:
 added license to github label patterns

Changes in file .github/workflows/Tests.yml:
 just a version update

Changes in file README.md:
 improved spacing a little

REMOVED file tests/check_codecov_regression
reactive-firewall added a commit that referenced this issue Aug 29, 2024
… WIP #46 & #49 & #50 -)

Changes in file .github/workflows/Tests.yml:
 - added extra tests for legacy build system as part of EOL and coverage support

Changes in file Makefile:
 - minor changes to conform with other changes

Changes in file pyproject.toml:
 update for new build system

Changes in file tests/check_integration_coverage:
 improved performance and stability

Changes in file tests/check_legacy_setup_coverage:
 new test case for legacy build system (setup.py)
reactive-firewall added a commit that referenced this issue Aug 29, 2024
  [COVERAGE] add additional coverage testing for legacy build system (- WIP #46 & #49 & #50 -)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CI Continuous Integration Tooling GitHub Anything Meta Testing Something can be verified
Projects
Status: Resolved
Development

No branches or pull requests

1 participant