Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cranelift-fuzzgen: Consume all trailing fuzz input #4862

Merged
merged 1 commit into from
Sep 7, 2022

Conversation

jameysharp
Copy link
Contributor

But don't keep going once we've consumed it all.

I got tired of having a bunch of copies of the same all-zeros test inputs at the end. This version always produces exactly one all-zeros test input at the end, which I feel okay about. I could remove that by explicitly checking for self.u.is_empty() but, eh, testing an all-zeros input could be useful.

This uses less of the fuzz input too since it doesn't need to come up with a count of how many test inputs to generate.

This changes the input format for this fuzz target, so I figure I'll bundle it together with #4824 and merge both sometime next week.

cc: @afonso360

@github-actions github-actions bot added the cranelift Issues related to the Cranelift code generator label Sep 3, 2022
@afonso360
Copy link
Contributor

I did some measurements on this.

When I first applied it roughly doubled the amount of runs that we do which is nice. (See the "Applying jameysharp recent patches" section on this comment)

Copy link
Contributor

@afonso360 afonso360 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! Thanks!

Comment on lines 144 to 150
// Continue generating input as long as we just consumed some of self.u. Otherwise we'll
// generate the same test input again on the next iteration, and there's no point testing
// it twice. Note that once self.u becomes empty we naturally don't consume any more of it,
// so this check covers that case as well as the case of testing a no-argument function.
// This lets us consume all remaining fuzz data.
while self.u.len() < last_len {
last_len = self.u.len();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be simpler as

loop {
    inputs.push(todo!("generate one input"));

    if u.is_empty() {
        break;
    }
}

which avoids the semi-subtle use of usize::MAX here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's how I wrote it first, actually! I could go back to loop plus a conditional break at the end if you want. But just testing u.is_empty doesn't work if generating an input doesn't consume any of u: it loops forever then. The usual way that would happen is if the generated function doesn't take any parameters, but I haven't ruled out the possibility that a parameter could only have one legal value and therefore consume no input. So instead I check that the loop makes some progress on every iteration, and stop after the first iteration where it doesn't make progress.

I agree that using usize::MAX here is subtle. We'd get nearly the same result from initializing last_len to self.u.len() + 1, if that would make you feel better?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we could also have a maximum length on inputs as well.

Otherwise I guess I would prefer making last_len an option and doing something like

while last_len.map_or(true, |last_len| self.u.len() < last_len) {
    ...
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm definitely not a fan of that alternative 😆 but I can still move the test to the end of the loop, if that works for you.

But don't keep going once we've consumed it all.
@jameysharp jameysharp merged commit b8b2fad into bytecodealliance:main Sep 7, 2022
@jameysharp jameysharp deleted the test-inputs-length branch September 7, 2022 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cranelift Issues related to the Cranelift code generator
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants