Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[fuzzer] Relax the FileCheck string for test fuzzer-custommutator.test (
#66343) The FileCheck string `LLVMFuzzerCustomMutatorLongSequence: {{.*}} MS: {{[0-9]*}} {{(([a-zA-Z]*-){11,})}} {{.*}}` is too restrictive and may fail the test in some case. If we look at the commit that added this check(66df989), This check is for printing out the long mutation sequence, such as this one ``` #53552 REDUCE cov: 6 ft: 6 corp: 5/9b lim: 4096 exec/s: 0 rss: 37Mb L: 2/3 MS: 54 ChangeByte-PersAutoDict-ChangeBit-ChangeBinInt-ChangeBit-ChangeBit-ChangeByte-CMP-EraseBytes-EraseBytes-CrossOver-InsertRepeatedBytes-ChangeByte-EraseBytes-InsertRepeatedBytes-ShuffleBytes-ChangeByte-ShuffleBytes-ChangeBit-CrossOver-ChangeBit-ShuffleBytes-ChangeBinInt-ShuffleBytes-EraseBytes-InsertByte-Custom-ShuffleBytes-CopyPart-InsertRepeatedBytes-PersAutoDict-InsertRepeatedBytes-ChangeByte-CrossOver-CrossOver-PersAutoDict-PersAutoDict-EraseBytes-ChangeBit-CopyPart-ChangeByte-CopyPart-InsertRepeatedBytes-CrossOver-CrossOver-CrossOver-CrossOver-ShuffleBytes-EraseBytes-InsertByte-InsertRepeatedBytes-CrossOver-EraseBytes-Custom- DE: "\377\377"-"\001\000"-"\001\000"-"\000\000\000\000\000\000\000\000"-"\001\000\000\000"- ``` But if we look at the code doing the printing ```cpp void MutationDispatcher::PrintMutationSequence(bool Verbose) { Printf("MS: %zd ", CurrentMutatorSequence.size()); size_t EntriesToPrint = Verbose ? CurrentMutatorSequence.size() : std::min(kMaxMutationsToPrint, CurrentMutatorSequence.size()); for (size_t i = 0; i < EntriesToPrint; i++) Printf("%s-", CurrentMutatorSequence[i].Name); if (!CurrentDictionaryEntrySequence.empty()) { Printf(" DE: "); EntriesToPrint = Verbose ? CurrentDictionaryEntrySequence.size() : std::min(kMaxMutationsToPrint, CurrentDictionaryEntrySequence.size()); for (size_t i = 0; i < EntriesToPrint; i++) { Printf("\""); PrintASCII(CurrentDictionaryEntrySequence[i]->GetW(), "\"-"); } } } ``` We can see that the `DE: XXX` is not always printed. So the following output is possible(and is from real-life failure), notince the missing of `DE: XXX`. ``` #13613 NEW cov: 5 ft: 5 corp: 4/6b lim: 4096 exec/s: 0 rss: 32Mb L: 2/2 MS: 27 InsertByte-ChangeBinInt-ChangeBinInt-CrossOver-ShuffleBytes-ChangeBit-EraseBytes-ShuffleBytes-InsertByte-InsertRepeatedBytes-CopyPart-InsertByte-ChangeByte-ChangeBit-InsertByte-CrossOver-EraseBytes-CopyPart-ShuffleBytes-EraseBytes-InsertByte-InsertRepeatedBytes-CrossOver-CrossOver-ShuffleBytes-ChangeBit-Custom- #13765 ...... ``` This output is totally legit and will fail that check. So I remove the check for the following strings, I think `MS: {{[0-9]*}} {{(([a-zA-Z]*-){11,})}}` is sufficient for checking the long mutation sequence. This should help resolve the flaky failure of fuzzer-custommutator.test.
- Loading branch information