Make uart_rx a verification component

[LarsAsplund]

This PR makes the uart_rx a VUnit verification component. It also means that there is a mechanism for making sure that all expected data has been seen before the testbench is allowed to exit. This means that the current limitation of just checking data that is received is removed and the bug causing data not to be emitted on UART1 is exposed.

@stnolting I would need help finding that bug but once that has been solved it would be harder to get it re-introduced now that the testbench is less forgiving.

[stnolting]

Awesome work! 👍

@stnolting I would need help finding that bug but once that has been solved it would be harder to get it re-introduced now that the testbench is less forgiving.

Tu be hones, I am a little bit lost here 😅
About what actual "bug" are we talking here? It seems like the Processor workflow "only" fails because there is a discrepancy in the actual test program output and the expected output. But that shouldn't be hard to fix?
Or am I missing something? 🤔

[LarsAsplund]

@stnolting There is an easy fix and a harder fix. I'm noticing that the testbench is expecting 37 tests when it should be 44. That's the easy fix. The harder fix is that the testbench is expecting the test result on UART1 after those initial NUL characters. Now the result is simply printed so the testbench keeps waiting until it times out.

[stnolting]

I'm noticing that the testbench is expecting 37 tests when it should be 44. That's the easy fix.

The problem is that the actual number of tests depends on the implemented hardware modules. For the testbench, all optional modules should be implemented. However, this number will change when new modules are introduced. But adapting a single line in the testbench should not be too hard 😉

The harder fix is that the testbench is expecting the test result on UART1 after those initial NUL characters. Now the result is simply printed so the testbench keeps waiting until it times out.

Ah ok, I think I now understand what you mean 😄
The test results are printed to the simulator console instead of the "real UART1 transmitter" but the testbench expects them as "real" UART output, right?

Printing the "whole" test result via the pysical UART takes quite some time in simulation. How about modifying the __neorv32_crt0_after_main function so that it just outputs an integer or hexadecimal value via the physical UART? The value could be the number of failed tests. So if the number is zero (or 0x00000000 to have a uniform string length) all tests passed.
All remaining UART outputs could still go to the simulator console if the SUPPRESS_OPTIONAL_UART_PRINT compile switch is set.

[LarsAsplund]

The problem is that the actual number of tests depends on the implemented hardware modules

That I can improve in the next PR. The expected number of tests can be an argument to the run script just like ci-mode.

Printing the "whole" test result via the pysical UART takes quite some time in simulation

Now it only outputs PRINT_CRITICAL. I thought the idea was that only emitting that on the UART would make the simulation fast enough

Nevertheless, just printing the number of passing test would be even faster. However, I think the total number of tests should be output as well. Otherwise it would be easy to add a test that doesn't work and also forgetting to increase that magic number in the testbench. The result would be a passing test.

[stnolting]

That I can improve in the next PR. The expected number of tests can be an argument to the run script just like ci-mode.

That would be great 👍

Now it only outputs PRINT_CRITICAL. I thought the idea was that only emitting that on the UART would make the simulation fast enough.
Nevertheless, just printing the number of passing test would be even faster. However, I think the total number of tests should be output as well. Otherwise it would be easy to add a test that doesn't work and also forgetting to increase that magic number in the testbench. The result would be a passing test.

So how about this:

• by setting SUPPRESS_OPTIONAL_UART_PRINT all UART output is masked. only CRITICAL will print via UART1 to the simulator console; there is no physical UART output at all (except for some nulls for the physical UART tests)
• the test's main function terminates; the after-main handler physically outputs a minimal test report via UART0 or UART1
• the minimal report might look like this: failed/total
  • 0/44 (decimal) or 0x00000000/0x0000002c (hexadecimal) edit decimal has less characters - so it is faster 😉

[LarsAsplund]

@stnolting That sounds reasonable. The important thing is that something is printed to the physical UART. That something should be significant enough to give some confidence that everything that needed to happen within the DUT, that we do not see, actually did happen. That's why I didn't like just printing the zero return code because a zero can easily be printed by mistake.

[stnolting]

Great! So 0/44 should be fine, right? And I think it should be output via UART1, since this happens to be our "critical" output channel, correct?

I will implement that as soon as possible - but before I have to fix a small bug I have found in the simulation output mode.
Furthermore, I would like to finish the first version of the Stream Link peripheral (#52)... I have way too many construction sides right now 😅

[LarsAsplund]

Correct!

[stnolting]

I have changed the test program in master to output a minimal report: FAILED_TESTS/TOTAL_TESTS. For the current test, this should be "0/45\n". I think we just need to rebase this and change the expected final report before this can be merged.

[LarsAsplund]

@stnolting Great! I rebased the PR and now all tests are green.