API access to perf results

[eddyashton]

I'm trying to call the Bencher REST API to access performance metrics directly, following the API docs here:

https://bencher.dev/docs/api/projects/perf/#get-v0projectsprojectperf

So far I've not been able to construct a valid URL.

First it seems all of the query parameters are required. Omitting any query parameter produces an error response like the following:

$ curl -i "https://api.bencher.dev/v0/projects/ccf/perf?testbeds=gha-virtual-ccf-sub&measures=Latency&branches=main"
HTTP/2 400 
content-type: application/json
x-request-id: 3ac82128-c228-42c9-9a3c-ddcd101a9e17
content-length: 131
date: Mon, 03 Jun 2024 13:57:00 GMT
server: Fly/63a16321 (2024-05-30)
via: 2 fly.io
fly-request-id: 01HZF676CD2MZV5ECQBJM58P3A-iad

{
  "request_id": "3ac82128-c228-42c9-9a3c-ddcd101a9e17",
  "message": "unable to parse query string: missing field `benchmarks`"
}

The project Path parameter is marked as Required - I think all of the Query parameters should also have this Required annotation.

But even when I provide all of them, I'm getting an error response that seems to be about url encoding:

$ curl -i "https://api.bencher.dev/v0/projects/ccf/perf?testbeds=gha-virtual-ccf-sub&benchmarks=commit_latency_ratio&measures=Latency&branches=main"
HTTP/2 400 
content-type: application/json
x-request-id: 9cd65d96-0789-4d7f-aa0a-26435c692206
content-length: 91
date: Mon, 03 Jun 2024 13:56:46 GMT
server: Fly/63a16321 (2024-05-30)
via: 2 fly.io
fly-request-id: 01HZF66RPD39N43DXX1EFD6B48-iad

{
  "request_id": "9cd65d96-0789-4d7f-aa0a-26435c692206",
  "message": "urlencoded: main"
}

As far as I can tell, the argument is correctly URL encoded. Other variations suggest the argument is being correctly decoded, but somehow it's still unhappy with the value:

$ curl "https://api.bencher.dev/v0/projects/ccf/perf?testbeds=gha-virtual-ccf-sub&benchmarks=commit_latency_ratio&measures=Latency&branches=%22main%22"
{
  "request_id": "36ce7979-1007-493e-b05d-2239d9ec23a5",
  "message": "urlencoded: \"main\""
}

$ curl "https://api.bencher.dev/v0/projects/ccf/perf?testbeds=gha-virtual-ccf-sub&benchmarks=commit_latency_ratio&measures=Latency&branches=6213%2Fmerge"
{
  "request_id": "0d8a476c-ae65-4981-9a24-df65001bb1de",
  "message": "urlencoded: 6213/merge"
}

Any idea what's going on there?

[epompeii]

@eddyashton sorry you're running into trouble.

You have to use the UUIDs for the branches, testbeds, benchmarks, and measures query params.

So something like this should work: https://api.bencher.dev/v0/projects/ccf/perf?branches=d5004f0a-5dbe-42bb-a821-1f55704d6ec2&testbeds=1e6f6a27-eb58-4f16-8d01-0148fbaed70e&benchmarks=3bae8305-29e0-4e5f-8157-01f8f471b408&measures=bc9fb376-9a85-478a-97fd-ebd7703c9663

The Bencher Console Perf Query UI query param API is a super set of the Bencher server API. So you can always just copy and paste everything after the ? in your URL and be able to hit (in your case) https://api.bencher.dev/v0/projects/ccf/perf with it.
The additional UI specific query params will just be ignored, if you don't trim them down. For example, this is the link that I started with above.

Great point on adding Required to the docs. I have created a tracking issue: #407