Analysisd - IT for the pre-decoder stage

[juliancnn]

Hi Team!.

To increase the integration test coverage, it is necessary to create tests for the pre-decoding stage of analysisd.
This stage currently supports the following syslog formats:

Syslog date format

• Dec 29 10:00:01
• 2015 Dec 29 10:00:01
• 2007-06-14T15:48:55-04:00 for syslog-ng isodate
• 2009-05-22T09:36:46.214994-07:00 for rsyslog
• 2015-04-16 21:51:02,805 for proftpd 1.3.5
• Mon Apr 17 18:27:14 2006 1 64.160.42.130 for xferlog date format
• 01/28-09:13:16.240702 for snort date format
• 01/28/1979-09:13:16.240702 for suricata (new) date format
• [Fri Feb 11 18:06:35 2004] [warn] for apache log format
• 1140804070.368 11623 seconds from 00:00:00 1970-01-01 UTC for squid date format
• 2021-04-21 10:16:09.404756-0700 for macos ULS --syslog output

Also check if month contains an umlaut. Umlauts are non-ASCII and use 2 slots in the char array
ex:

Mär 02 17:30:52

Regards,
Julian

[mauromalara]

@danisan90 It is necessary to test this against the logtest socket, you can see use examples within

tests/integration/test_logtest

You can check it manually as follows:

1. Execute logtest in debug mode

/var/ossec/bin/wazuh-logtest -d

2. Paste a valid log (with the format described in this issue). Ex:

Oct 15 21:07:56 linux-agent sshd[29205]: Invalid user blimey from 18.18.18.18 port 48928

3. Check in the JSON output the "predecoder" key within the "output" key.

"output":{
      "predecoder":{
         "program_name":"sshd",
         "timestamp":"Oct 15 21:07:56",
         "hostname":"linux-agent"
      }
}

4. This should have a program_name and a correct timestamp.

[danisan90]

The working branch for this issue is 1498-predecoder-stage-test.

[mauromalara]

Style guide

pycodestyle --max-line-length=120 tests/integration/test_analysisd/test_predecoder_stage/test_predecoder_stage.py (FAILED) 🔴

Results

tests/integration/test_analysisd/test_predecoder_stage/test_predecoder_stage.py:87:1: W293 blank line contains whitespace
tests/integration/test_analysisd/test_predecoder_stage/test_predecoder_stage.py:113:81: E231 missing whitespace after ','
tests/integration/test_analysisd/test_predecoder_stage/test_predecoder_stage.py:113:121: E501 line too long (255 > 120 characters)

QA-docs execution

test_predecoder_stage

qa-docs --tests-path /home/mauro/Documents/Wazuh/issue/1498/wazuh-qa/tests/integration/test_analysisd/test_predecoder_stage/ -t test_predecoder_stage

Results

2022-01-17 16:38:59,066 - INFO - Looking for test_predecoder_stage.py
2022-01-17 16:38:59,066 - INFO - Parsing the following test(s) ['test_predecoder_stage']
2022-01-17 16:38:59,076 - INFO - Running QADOCS
2022-01-17 16:38:59,076 - INFO - Looking for test_predecoder_stage.py
2022-01-17 16:38:59,225 - INFO - Run completed, documentation location: /tmp/qa_docs/output

Output File

test_predecoder_stage.zip

[mauromalara]

Style guide

• Command: pycodestyle --max-line-length=120 tests/integration/test_analysisd/test_predecoder_stage/test_predecoder_stage.py
• Result: PASSED 🟢