Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor VDT integration tests: feeds and scan types #2650

Merged
merged 100 commits into from
Mar 9, 2022

Conversation

jmv74211
Copy link
Contributor

@jmv74211 jmv74211 commented Mar 9, 2022

Related issue
close #2592
close #2639

Description

This PR adds the test refactor to the test feeds, and the scan types. It also adds new feed organization, and standardizes all VDT test files.

The changes made are as follows

Changes in tests

Refactored test modules

  • test_feeds/test_cpe_indexing.py
  • test_feeds/test_download_feeds.py
  • test_feeds/test_duplicate_feeds.py
  • test_feeds/test_import_invalid_feed_type.py
  • test_general_settings/test_interval.py
  • test_scan_results/test_scan_nvd.py
  • test_scan_types/test_baseline_scan_type.py
  • test_scan_types/test_full_scan_type.py
  • test_scan_types/test_partial_scan_type.py

Test modules added

  • test_vulnerability_inventory/test_vulnerability_inventory_baseline_scan.py
  • test_vulnerability_inventory/test_vulnerability_inventory_full_scan.py
  • test_vulnerability_inventory/test_vulnerability_inventory_partial_scan.py

Tests eliminated

  • test_feeds/test_extra_tags_*_feed.py
  • test_feeds/test_invalid_syntax_*_feed.py
  • test_feeds/test_invalid_values_*_feed.py
  • test_feeds/test_missing_tags_*_feed.py

Changes in the framework

Added

  • Add new global variables to the wazuh_testing module.
  • Add new methods and documentation to the agent_db module.
  • Add new methods and documentation to the global_db module.
  • Add information and mocking of systems to the mocking module.
  • Add new methods to the mocking module.
  • Add new utility functions to vulnerability detector module
  • Add new event monitoring functions, and removes deprecated ones.
  • Add new function to time module to parse datetimes from feeds.
  • Add new fixtures for mocking agents with custom systems.
  • Add new fixtures for vulnerability detector
  • Add autoconfiguration of local_internal_options for VDT tests.
  • Add new custom feeds for each provider OS
  • Add new custom file with vulnerable packages info.
  • Add tiers to all VDT tests

Improved

  • Improve agent mockeying process
  • Improve documentation of all VDT tests.

Deleted

  • Remove many variables and deprecated methods from the vulnerability detector module.
  • Remove utils module from vulnerability detector.

Tests

  • Proven that tests pass when they have to pass.
  • Proven that tests fail when they have to fail.

Checks

The following checks have been proven with the following package

Version Revision Repository
4.4.0 0.40400.20220307 staging
  • Passed 5 rounds of VDT ITs in Jenkins
Result Build URL
🟢 https://ci.wazuh.info/job/Test_integration/22890/
🟢 https://ci.wazuh.info/job/Test_integration/22891/
🟢 https://ci.wazuh.info/job/Test_integration/22892/
🟢 https://ci.wazuh.info/job/Test_integration/22893/
🟢 https://ci.wazuh.info/job/Test_integration/22894/
  • Passed 5 rounds of VDT ITs in local
Round 1 🟢
======================== 185 passed, 1 skipped, 1 xfailed in 1841.35s (0:30:41) ========================
Round 2 🟢
======================== 185 passed, 1 skipped, 1 xfailed in 1841.35s (0:30:41) ========================
Round 3 🟢
======================== 185 passed, 1 skipped, 1 xfailed in 1841.35s (0:30:41) ========================
Round 4 🟢
======================== 185 passed, 1 skipped, 1 xfailed in 1841.35s (0:30:41) ========================
Round 5 🟢
======================== 185 passed, 1 skipped, 1 xfailed in 1841.35s (0:30:41) ========================
  • All nightly integration tests work
Test Result
Analysisd IT - Tier 0 - manager 🟢
Analysisd IT - Tier 1 - manager 🟢
AR IT - Tier 0 - manager 🟢
AR IT - Tier 1 - centos_agent 🟢
Authd IT - Tier 0 - manager 🟢
Authd IT - Tier 1 - manager 🟢
FIM IT - Tier 0 - centos_agent 🟢
FIM IT - Tier 0 - manager 🟢
FIM IT - Tier 1 - centos_agent 🟢
FIM IT - Tier 1 - manager 🟢
Logtest IT - Tier 0 - manager 🟢
Remoted IT - Tier 0 - manager 🟢
Remoted IT - Tier 1 - manager 🟢
Rids IT - Tier 0 - manager 🟢
Rootcheck IT - Tier 0 - manager 🟢
Rootcheck IT - Tier 1 - manager 🟢
Vul. Det. IT - Tier 0 - manager 🟢
Vul. Det. IT - Tier 1 - manager 🟢
Wazuh DB IT - Tier 0 - manager 🟢

fedepacher and others added 30 commits February 9, 2022 20:06
Due to the new feed structure changes
…ility-removal

Vulnerability detector test module refactor: `test_alert_vulnerability_removal`
Vulnerability detector test module refactor: test_scan_nvd_feed
…te-vulns

Vulnerability detector test module refactor: `test_duplicate_feeds`
jmv74211 and others added 23 commits March 3, 2022 16:55
Vulnerability detector test module refactor: `test_validate_feed_content`
Necessary due to compatibilities issues, since datetime.fromisoformat needs python >=3.7.0
Add new logic to check if a downloaded feed has been recently updated in `test_downloads_feeds` module
@jmv74211 jmv74211 self-assigned this Mar 9, 2022
@jmv74211 jmv74211 requested a review from Rebits March 9, 2022 10:13
@jmv74211 jmv74211 merged commit b61e90c into master Mar 9, 2022
@jmv74211 jmv74211 deleted the 2639-vdt-refactor-improvements branch March 9, 2022 10:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants