This action publishes results from BenchmarkDotNet benchmarks to a GitHub repository.
This action was inspired by the benchmark-action/github-action-benchmark action.
The action generates summarised results from BenchmarkDotNet's JSON output files
(such as output with --exporters json
) and writes a summary of them to a JSON file. The results are then
pushed to a Git branch in the specified repository.
From there, you can then use the results as you please, such as to generate a dasboard to track the performance of your code over time. An easy way to achieve this is with a static GitHub Pages site and some JavaScript to render the results.
A repository that demonstrates this approach can be found here: martincostello/benchmarks-demo.
Warning
Do not use this action with pull requests that come from untrusted sources, such as from public forks or Dependabot. A malicious user could craft a pull request that hijacks the permissions of the configured GitHub token. See this GitHub Security blog post for more information.
Below is an example of a full GitHub Actions workflow to run the benchmarks in a .NET
project and then publish the results to the gh-pages
branch of the same GitHub repository.
Important
If pushing to the same repository, ensure that you do not create a circular workflow that triggers itself when the results are pushed to the repository.
name: benchmark
on:
push:
branches: [ main ]
permissions:
contents: read
jobs:
benchmark:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
- run: dotnet run --project Benchmarks.csproj --configuration Release
- uses: martincostello/benchmarkdotnet-results-publisher@v1
If you wish to publish the results to a different repository, you can specify the repository and access token to use as shown in the example below.
Important
GITHUB_TOKEN
cannot be used to push results to a different repository.
You must use a personal access token or a token for a GitHub App instead.
name: benchmark
on:
push:
branches: [ main ]
permissions:
contents: read
jobs:
benchmark:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-dotnet@v4
- run: dotnet run --project Benchmarks.csproj --configuration Release
- uses: martincostello/benchmarkdotnet-results-publisher@v1
with:
repo: '${{ github.repository_owner }}/benchmarks' # Publish to a different repository
repo-token: ${{ secrets.BENCHMARKS_TOKEN }} # Use a secret for the access token
This action is useful for projects that use BenchmarkDotNet to run benchmarks and want to publish the results to a GitHub repository.
For example, I use this action in various repositories of my own, which publish the results of their benchmarks to my benchmarks repository. That data from that repository is then consumed by a GitHub Pages site to display the results using static HTML pages and Plotly.
- Publishes the results of BenchmarkDotNet benchmarks to a Git branch in a GitHub repository.
- Write the Markdown summary of the BenchmarkDotNet results to the workflow run summary.
- Optionally fail the workflow if the duration or memory threshold is exceeded for any benchmark using.
- Optionally posts a comment to a commit or pull request if the duration or memory threshold is exceeded for any benchmark (example).
- Optionally truncate the number of results for each suite of benchmark results to a fixed sliding window.
Tip
This action is only intended to be used to track the relative performance of your benchmarks over time when using GitHub-hosted runners. For more stable results or for tracking absolute performance, consider using a self-hosted runner to run your benchmarks.
Name | Description | Default |
---|---|---|
branch |
The optional Git branch to push the results to. | gh-pages |
comment-on-threshold |
Whether to post a comment if either the duration or memory threshold is exceeded for any benchmark. | false |
commit-message |
The optional Git commit message to use. | - |
commit-message-prefix |
The optional prefix to use when generating Git commit messages if a custom commit message is not used. | - |
fail-on-threshold |
Whether to cause the workflow to fail if either the duration or memory threshold is exceeded for any benchmark. | false |
fail-threshold-duration |
The optional threshold, as a ratio, which determines if the current workflow fails based on the previous result for a duration metric. | 2 (i.e. 200%) |
fail-threshold-memory |
The optional threshold, as a ratio, which determines if the current workflow fails based on the previous result for a memory usage metric. | 2 (i.e. 200%) |
max-items |
The optional maximum number of results to include for each suite of benchmark results. | Unlimited |
name |
The optional name to use to group the benchmark results that are found into suites. | Inferred from BenchmarkDotNet results |
output-file-path |
The optional path of the file to write the results to. | ./data.json |
output-step-summary |
Whether to output the Markdown results from BenchmarkDotNet to GITHUB_STEP_SUMMARY . |
true |
repo |
The optional GitHub repository to push the results to. | github.repository |
repo-token |
The GitHub access token to use to push the results to a GitHub repository. | github.token |
results-path |
The optional path of the BenchmarkDotNet results directory to process. | ./BenchmarkDotNet.Artifacts |
None.
This action requires the following permissions, depending on the features used:
contents: write
- Required to read and write the results to the target repository.issues: write
- Required to post comments ifcomment-on-threshold
is enabled.
Any feedback or issues can be added to the issues for this project in GitHub.
The repository is hosted in GitHub: https://github.com/martincostello/benchmarkdotnet-results-publisher.git
This project is licensed under the Apache 2.0 license.