Using --format=json with --nocapture

[mchernyavsky]

I'm looking for a way to get test results, including stdout/stderr, in json format.

I've tried to use --format=json with --nocapture and have got an unexpected result, imo.

Example file:

#[cfg(test)]
mod tests {
    #[test]
    fn test1() {
        println!("Hello from test #1");
        panic!();
    }
    #[test]
    fn test2() {
        println!("Hello from test #2");
    }
}

1. cargo test -- -Z unstable-options --format=json output:

    Finished dev [unoptimized + debuginfo] target(s) in 0.03s                                                                                                                        
     Running target/debug/deps/rust_sandbox-3aca71fd58cbc041
{ "type": "suite", "event": "started", "test_count": "0" }
{ "type": "suite", "event": "ok", "passed": 0, "failed": 0, "allowed_fail": 0, "ignored": 0, "measured": 0, "filtered_out": "0" }
     Running target/debug/deps/tests-03a860f74891dd25
{ "type": "suite", "event": "started", "test_count": "2" }
{ "type": "test", "event": "started", "name": "tests::test1" }
{ "type": "test", "event": "started", "name": "tests::test2" }
{ "type": "test", "name": "tests::test2", "event": "ok" }
{ "type": "test", "name": "tests::test1", "event": "failed", "stdout": "Hello from test #1\nthread 'tests::test1' panicked at 'explicit panic', tests/tests.rs:6:9\nnote: Run with `RUST_BACKTRACE=1` for a backtrace.\n" }
{ "type": "suite", "event": "failed", "passed": 1, "failed": 1, "allowed_fail": 0, "ignored": 0, "measured": 0, "filtered_out": "0" }
error: test failed, to rerun pass '--test tests'

2. cargo test -- --nocapture -Z unstable-options --format=json output:

    Finished dev [unoptimized + debuginfo] target(s) in 0.03s                                                                                                                        
     Running target/debug/deps/rust_sandbox-3aca71fd58cbc041
{ "type": "suite", "event": "started", "test_count": "0" }
{ "type": "suite", "event": "ok", "passed": 0, "failed": 0, "allowed_fail": 0, "ignored": 0, "measured": 0, "filtered_out": "0" }
     Running target/debug/deps/tests-03a860f74891dd25
{ "type": "suite", "event": "started", "test_count": "2" }
{ "type": "test", "event": "started", "name": "tests::test1" }
{ "type": "test", "event": "started", "name": "tests::test2" }
Hello from test #1
Hello from test #2
thread 'tests::test1' panicked at 'explicit panic', tests/tests.rs:6:9
note: Run with `RUST_BACKTRACE=1` for a backtrace.
{ "type": "test", "name": "tests::test2", "event": "ok" }
{ "type": "test", "name": "tests::test1", "event": "failed" }
{ "type": "suite", "event": "failed", "passed": 1, "failed": 1, "allowed_fail": 0, "ignored": 0, "measured": 0, "filtered_out": "0" }
error: test failed, to rerun pass '--test tests'

I expected to see this happen:

• In the 1st case, the Hello from test #1 line doesn't appear in the stdout field.
• In the 2nd case, the println! output and errors are recorded in different fields (e.g. stdout and stderr).

Expected behavior can be useful for integrating test frameworks with IDEs and editors.

I'm using:

• rustc 1.29.1 (b801ae6 2018-09-20)