Skip to content
This repository has been archived by the owner on Jan 12, 2020. It is now read-only.

Use fuzzing to drive property testing #7

Merged
merged 5 commits into from
Sep 9, 2018
Merged

Use fuzzing to drive property testing #7

merged 5 commits into from
Sep 9, 2018

Commits on Aug 25, 2018

  1. Initial work toward using a fuzzer as a QC tool

    There's some problems with the available QC tools, at least for the
    purposes of this project. Namely:
    
     * the Rust testing framework limits the amount of concurrency
       available, starving some tests of runtime
     * there is no distribution available
     * test-case generation can be slow
    
    Considering we're randomly searching through a state space the name of
    the game is speed. To that end, I have in mind a notion of
    QuickChecking which operates first in fuzzer mode--quickly throwing
    random nonsense into your program to find crashes, track branch
    conditions in the program--and, once a crash has been found, switches
    into QC mode to shrink the found case. This commit does not have that
    mode operation notion, only shuffles code around to run a property
    test under AFL. There is no shrinking.
    
    Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
    blt committed Aug 25, 2018
    Configuration menu
    Copy the full SHA
    2873f6e View commit details
    Browse the repository at this point in the history
  2. Adapt approach to honggfuzz, avoid allocation crashes

    This commit adjust the fuzzer from AFL to honggfuzz. Honggfuzz has the
    advantage of being multi-threaded by default but, honestly, the
    initial big draw was easy hook-in to lldb. The AFL fuzzer from the
    previous commit 'detected' HashMap panicing when we requested too much
    capacity but I found that an easy call to "cargo hfuzz run-debug
    hash_map" made the failure much more clear.
    
    The main change here is around the Reserve operation, limited now to a
    smallish reservation bump. This avoids the case where HashMap will
    panic on call to its `reserve`, being unable to signal that its
    allocation failed. The model is also careful about requested too much
    capacity, as noted. If HashMap published its overhead per pair we
    could calculate when the allocations would fail but it, reasonably,
    does not publish such information.
    
    Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
    blt committed Aug 25, 2018
    Configuration menu
    Copy the full SHA
    9596448 View commit details
    Browse the repository at this point in the history
  3. Resolve test build failure

    I neglected to remove quickcheck from `lib.rs`, though it is no longer
    a project dependency. This caused test build of the project to fail.
    
    Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
    blt committed Aug 25, 2018
    Configuration menu
    Copy the full SHA
    ad45847 View commit details
    Browse the repository at this point in the history

Commits on Sep 9, 2018

  1. Introduce FiniteBuffer

    This commit introduces FiniteBuffer into the hash map
    fuzz. FiniteBuffer, unlike RingBuffer, does not produce an infinite
    amount of data but will limit production to the total number of bytes
    available in `data`. This change means we have to guard each Arbitrary
    and it's possible that if the fuzzer is not correctly configured we'll
    only check small total operations. But, it is quite a bit faster and
    fuzzers that do minimization -- like AFL -- are able to eat into the
    shrink step needed for proper quickchecking.
    
    This is promising, I think.
    
    Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
    blt committed Sep 9, 2018
    Configuration menu
    Copy the full SHA
    41a8612 View commit details
    Browse the repository at this point in the history
  2. Back off aggressive use of clippy

    Clippy has changed its lint names to scoped form. This plays with
    CI pipelines using stable release until 'tool_lint' feature lands.
    Seems like this is targeted for the 2018 edition so, uh, this
    change will surely be reverted in the near term.
    
    Signed-off-by: Brian L. Troutwine <brian@troutwine.us>
    Brian L. Troutwine authored and blt committed Sep 9, 2018
    Configuration menu
    Copy the full SHA
    1e8544d View commit details
    Browse the repository at this point in the history