-
Notifications
You must be signed in to change notification settings - Fork 52
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Extend project-summary API endpoint to include project wide summary #1323
Comments
the
The preferred, and much appreciated, way would be to submit patches to the webapp and provide APIs that expose this data. We'd also be happy to keep this stable. Basically, web_db_creator_from_summary converts the |
That said, if this is specifically related to Could you elaborate a bit more what you have in mind here? Is it something we could add to the fuzzers overview table as well? |
Do you have in mind stuff such as checking intersections/specific differences of reachability/coverage between fuzzers? In the |
For reference, the oss-fuzz-scanner goes through the cloud storage and interprets the I'd probably prefer to focus on the API, and the API kind of arose from that project. However, for users that want to come up with interesting/arbitrary analysis we may simply end up with one specific API that offers "everything available" and then this just does the job of assembling all introspector data from all projects in one accessible point. I guess the downside is that we're not really against adding arbitrarily many APIs (within reason) so adding APIs that may be of value to others is nice -- but, having all data available makes it quiet nice to experiment and "explore" possibilities. Hmm. Would this fit your visions? |
Easy, I'll try and find the time over the next week or two for that.
So for some context I often participate in the google/bughunters oss-fuzz rewards program, for a bit of extra-cash on the side. I think you've reviewed a number of my PR's over at google/oss-fuzz. I'm trying to optimize my workflow a little bit and create somewhat of a hybrid between;
So being able to collect those metrics programmatically would allow me to ask broader questions like.
So yeah some of these changes would likely fit nicely into UI. The only other thing that I'd add is that I could see myself calling the API asynchronously for each project in oss-fuzz. So having some guidelines on rate-limiting would be great.
Honestly haven't thought too much about this. But it does spark up some other thoughts. I have been interested in working out a way to automatically identify the "purpose" of a fuzz-harness. e.g. which top-level function in the fuzz-harness has the most reachable complexity. But I haven't fully fleshed this thought out yet.
Cool I'll have a look into that when I get to extending the project-summary endpoint.
Yeah for sure, having all the raw data available would be great for experimenting with things, and probably my ideal situation. I also think there is a decent amount of derivative data that would also be super useful e.g. optimal target analysis. Perhaps it would make sense to do something like a daily datadump rather than a complete API? Or even storing all the data as a public bigquery dataset. The latter would be easier to experiment/play with the data. Not sure which would be a better approach, but I could see myself hammering the entire API surface once a day for each project (within rate-limits). |
Got it -- the target oracle was meant to this kind of use case (i.e. help finding the optimal spot between value gained and effort needed amongst all OSS-Fuzz projects) so great it sparked your interest! I think I have a good understanding of what you're looking for and will push for this in the coming days, and assess what options we have for easily making all data available. Also yeah definitely saw your contributions on OSS-Fuzz, great work! |
This makes all introspector data available through the API, which can be useful for experimenation and exploration. Discussed in #1323 Ref: #1323 (comment) Signed-off-by: David Korczynski <david@adalogics.com>
This makes all introspector data available through the API, which can be useful for experimenation and exploration. Discussed in #1323 Ref: #1323 (comment) Signed-off-by: David Korczynski <david@adalogics.com>
At the moment the project-summary endpoint provides a per fuzz-harness summary rather than a project wide summary. i.e. it provides reachable cyclomatic complexity/line coverage per fuzz-harness. It would be great if we could analyse each fuzzer relative to the complete project statistics. Essentially being able to reconstruct something like this using the API would be great.
I've been perusing the s3 bucket that stores all the introspector data, and I think that the data is already available here. Forming a sort of undocumented and (probably) unintentional API. Note that the fields for cyclomatic complexity match the screenshot.
If I was to build an API client around that would it be stable? Or would it be better to formally include this in the public API?
The text was updated successfully, but these errors were encountered: