Vulnerability detector test module refactor: test_duplicate_feeds

[fedepacher]

Related issue
#2525

Description

This PR adds changes and enhancements to the testing framework for vulnerability detector.

The changes made are as follows:

Added

• Add new functions to check log events.
• Add testing module: test_duplicate_feeds and data.

Improvements

• Added a new structure to the test.
• Refactor the test in order to get PASSED.

Tests

• All vulnerability detector tests
• test_alert_vulnerability_removal

Outputs

All vulnerability detector tests

======================================= test session starts =======================================
platform linux -- Python 3.6.8, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: /mnt/qa/tests/integration, configfile: pytest.ini
plugins: metadata-1.11.0, testinfra-5.0.0, html-3.1.1
collected 2580 items                                                                              

test_vulnerability_detector/test_feeds/test_download_feeds.py ssssssssssssssss              [  0%]
test_vulnerability_detector/test_feeds/test_invalid_type_custom_feeds.py ssssssssssssssssss [  1%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [  4%]
sssssssss                                                                                   [  5%]
test_vulnerability_detector/test_feeds/test_invalid_type_url_feeds.py sssssssssssssssssssss [  6%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [  9%]
ssssss                                                                                      [  9%]
test_vulnerability_detector/test_feeds/test_validate_feed_content.py ...................... [ 10%]
................                                                                            [ 11%]
test_vulnerability_detector/test_feeds/alas/test_extra_tags_alas_feed.py ssssssssssssssssss [ 11%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 15%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 18%]
ssssssssssssssssssssssssssssssssssssssssssss                                                [ 20%]
test_vulnerability_detector/test_feeds/alas/test_invalid_syntax_alas_feed.py ssssssssssssss [ 21%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 24%]
sssss                                                                                       [ 24%]
test_vulnerability_detector/test_feeds/alas/test_invalid_values_alas_feed.py ssssssssssssss [ 25%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 29%]
sssssssssssssssss                                                                           [ 29%]
test_vulnerability_detector/test_feeds/alas/test_missing_tags_alas_feed.py ssssssssssssss   [ 30%]
test_vulnerability_detector/test_feeds/archlinux/test_extra_tags_archlinux_feed.py ssssssss [ 30%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 34%]
sssssssssssssssssssssss                                                                     [ 34%]
test_vulnerability_detector/test_feeds/archlinux/test_invalid_syntax_archlinux_feed.py ssss [ 35%]
sssssssssssssssssssssssssssssssssssssssssssssssssss                                         [ 37%]
test_vulnerability_detector/test_feeds/archlinux/test_invalid_values_archlinux_feed.py ssss [ 37%]
sssssssssssssssss                                                                           [ 37%]
test_vulnerability_detector/test_feeds/archlinux/test_missing_tags_archlinux_feed.py ssssss [ 38%]
sssss                                                                                       [ 38%]
test_vulnerability_detector/test_feeds/canonical/test_extra_tags_canonical_feed.py ssssssss [ 38%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 42%]
sssssssssssssssssssssss                                                                     [ 43%]
test_vulnerability_detector/test_feeds/canonical/test_invalid_syntax_canonical_feed.py ssss [ 43%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss             [ 46%]
test_vulnerability_detector/test_feeds/canonical/test_invalid_values_canonical_feed.py ssss [ 46%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 49%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss                [ 52%]
test_vulnerability_detector/test_feeds/canonical/test_missing_tags_canonical_feed.py ssssss [ 53%]
sssssssssssssssssssssssssss                                                                 [ 54%]
test_vulnerability_detector/test_feeds/debian/test_extra_tags_debian_feed.py ssssssssssssss [ 54%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 58%]
sssssssssssssssss                                                                           [ 58%]
test_vulnerability_detector/test_feeds/debian/test_invalid_syntax_debian_feed.py ssssssssss [ 59%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss                   [ 62%]
test_vulnerability_detector/test_feeds/debian/test_invalid_values_debian_feed.py ssssssssss [ 62%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss                     [ 65%]
test_vulnerability_detector/test_feeds/debian/test_missing_tags_debian_feed.py ssssssssssss [ 65%]
ssssssssssssssssssssssssss                                                                  [ 66%]
test_vulnerability_detector/test_feeds/msu/test_extra_fields_msu_feed.py ssssssssssssssssss [ 67%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 70%]
sssssssssssss                                                                               [ 71%]
test_vulnerability_detector/test_feeds/msu/test_invalid_syntax_msu_feed.py ssssssssssssssss [ 72%]
sssssssssssssssssssssssssssssssssssssssss                                                   [ 73%]
test_vulnerability_detector/test_feeds/msu/test_invalid_values_msu_feed.py ssssssssssssssss [ 74%]
sssssssssssssssssssssssssssssssssssssssssssss                                               [ 76%]
test_vulnerability_detector/test_feeds/msu/test_missing_fields_msu_feed.py sssssss          [ 76%]
test_vulnerability_detector/test_feeds/redhat/test_extra_fields_redhat_feed.py ssssssssssss [ 76%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 80%]
sssssssssssssssssss                                                                         [ 81%]
test_vulnerability_detector/test_feeds/redhat/test_invalid_syntax_redhat_feed.py ssssssssss [ 81%]
ssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss                    [ 84%]
test_vulnerability_detector/test_feeds/redhat/test_invalid_values_redhat_feed.py ssssssssss [ 84%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssssss [ 88%]
sssssssssssssssssssssssssssssssssssssssssssssssssssssssssss                                 [ 90%]
test_vulnerability_detector/test_feeds/redhat/test_missing_fields_redhat_feed.py ssssssssss [ 90%]
sssssssssssssssss                                                                           [ 91%]
test_vulnerability_detector/test_general_settings/test_enabled.py ..                        [ 91%]
test_vulnerability_detector/test_general_settings/test_interval.py ............             [ 92%]
test_vulnerability_detector/test_general_settings/test_min_full_scan_interval.py .          [ 92%]
test_vulnerability_detector/test_general_settings/test_retry_interval.py ..                 [ 92%]
test_vulnerability_detector/test_general_settings/test_run_on_start.py ..                   [ 92%]
test_vulnerability_detector/test_providers/test_enabled.py ................................ [ 93%]
................................                                                            [ 94%]
test_vulnerability_detector/test_providers/test_missing_os.py ..............                [ 95%]
test_vulnerability_detector/test_providers/test_multiple_provider_feeds.py ....             [ 95%]
test_vulnerability_detector/test_providers/test_os.py ................................      [ 96%]
test_vulnerability_detector/test_providers/test_update_from_year.py ..........              [ 97%]
test_vulnerability_detector/test_providers/test_update_interval.py ......                   [ 97%]
test_vulnerability_detector/test_scan_results/test_alas_inventory_alas_feed.py ss           [ 97%]
test_vulnerability_detector/test_scan_results/test_alert_vulnerability_removal.py ..        [ 97%]
test_vulnerability_detector/test_scan_results/test_archlinux_inventory_archlinux_feed.py ss [ 97%]
                                                                                            [ 97%]
test_vulnerability_detector/test_scan_results/test_debian_inventory_debian_feed.py ss       [ 97%]
test_vulnerability_detector/test_scan_results/test_duplicate_feeds.py ......                [ 97%]
test_vulnerability_detector/test_scan_results/test_macos_inventory.py ss                    [ 97%]
test_vulnerability_detector/test_scan_results/test_msu_inventory_msu_feed.py sss            [ 97%]
test_vulnerability_detector/test_scan_results/test_redhat_inventory_redhat_feed.py ssss     [ 98%]
test_vulnerability_detector/test_scan_results/test_scan_different_cves.py ssssssssssss      [ 98%]
test_vulnerability_detector/test_scan_results/test_scan_nvd_feed.py ssssssssssssssss        [ 99%]
test_vulnerability_detector/test_scan_results/test_scan_providers_and_nvd_feed.py sssssssss [ 99%]
sss                                                                                         [ 99%]
test_vulnerability_detector/test_scan_results/test_ubuntu_inventory_canonical_feed.py ssss  [ 99%]
test_vulnerability_detector/test_scan_types/test_baseline_scan_type.py s                    [ 99%]
test_vulnerability_detector/test_scan_types/test_full_scan_type.py .                        [ 99%]
test_vulnerability_detector/test_scan_types/test_partial_scan_type.py s                     [ 99%]
test_vulnerability_detector/test_windows/test_cpe_indexing.py s                             [100%]

========================= 196 passed, 2384 skipped in 2090.92s (0:34:50) ==========================

test_alert_vulnerability_removal

========================== 64 passed in 583.83s (0:09:43) ==========================
[root@manager-vdt-01 integration]# python3 -m pytest test_vulnerability_detector/test_scan_results/test_duplicate_feeds.py 
=============================== test session starts ================================
platform linux -- Python 3.6.8, pytest-6.2.3, py-1.10.0, pluggy-0.13.1
rootdir: /mnt/qa/tests/integration, configfile: pytest.ini
plugins: metadata-1.11.0, testinfra-5.0.0, html-3.1.1
collected 6 items                                                                  

test_vulnerability_detector/test_scan_results/test_duplicate_feeds.py ...... [100%]

=========================== 6 passed in 86.42s (0:01:26) ===========================

[jmv74211]

Why you use here this custom timeout, and not in the other cases?

[fedepacher]

Because the second time I check for the Finish... message It fails for the timeout exception. This situation did not happen in the first call of the function.