Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Refchecker to Tests #9862

Merged
merged 4 commits into from
Dec 7, 2022
Merged

Conversation

Mab879
Copy link
Member

@Mab879 Mab879 commented Nov 22, 2022

Description:

  • Fix utils/refchecker.py
  • Add utils/refchecker.py checks for RHEL7 and RHEL8 CIS profiles

Rationale:

  • Make utils/refchecker.py useful
  • Help find bugs sooner

Review Hints:

  • Remove the CIS keys from a file like linux_os/guide/system/software/integrity/crypto/configure_crypto_policy/rule.yml, and it should make the tests fail.

@Mab879 Mab879 added Infrastructure Our content build system Test Suite Update in Test Suite. labels Nov 22, 2022
@Mab879 Mab879 added this to the 0.1.66 milestone Nov 22, 2022
@github-actions
Copy link

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment
Open in Gitpod

Oracle Linux 8 Environment
Open in Gitpod

Copy link
Collaborator

@jan-cerny jan-cerny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have run some of the tests on the RHEL 8 content and when I intentionally removed some CIS references from rules the test started to fail as is expected.

tests/CMakeLists.txt Outdated Show resolved Hide resolved
tests/CMakeLists.txt Outdated Show resolved Hide resolved
utils/refchecker.py Outdated Show resolved Hide resolved
@Mab879
Copy link
Member Author

Mab879 commented Nov 29, 2022

/packit build

@marcusburghardt
Copy link
Member

@jan-cerny , I saw you already reviewed this PR once. Are you comfortable to assign it to yourself?

@openshift-merge-robot openshift-merge-robot added the needs-rebase Used by openshift-ci bot. label Dec 2, 2022
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Used by openshift-ci bot. label Dec 2, 2022
@jan-cerny jan-cerny self-assigned this Dec 5, 2022
Copy link
Collaborator

@jan-cerny jan-cerny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for the delay.

The quiet mode isn't good because if the references break and the tests start to fail people can't see which rules fail the test. Don't use the quiet mode or make the errors visible also in quiet mode.

Also please resolve the conflicts.

The script was not using the latest changes in the project
and did not run.
Made refchecker use full paths for rule_dir_json,
build config, and controls.
Before the first line was being overriden.
@codeclimate
Copy link

codeclimate bot commented Dec 5, 2022

Code Climate has analyzed commit 9994c38 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 100.0% (50% is the threshold).

This pull request will bring the total coverage in the repository to 48.8% (0.0% change).

View more on Code Climate.

@Mab879 Mab879 requested a review from jan-cerny December 5, 2022 18:51
Copy link
Collaborator

@jan-cerny jan-cerny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have run the tests locally and if I break something they start to fail as expected. Also, I have seen that they run in GitHub Actions.

@jan-cerny jan-cerny merged commit 46a3a0a into ComplianceAsCode:master Dec 7, 2022
@Mab879 Mab879 deleted the refchecker branch December 7, 2022 13:34
This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Infrastructure Our content build system Test Suite Update in Test Suite.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants