BenchmarkDotNet Results Publisher

This action publishes results from BenchmarkDotNet benchmarks to a GitHub repository.

This action was inspired by the benchmark-action/github-action-benchmark action.

The action generates summarised results from BenchmarkDotNet's JSON output files (such as output with --exporters json) and writes a summary of them to a JSON file. The results are then pushed to a Git branch in the specified repository.

From there, you can then use the results as you please, such as to generate a dasboard to track the performance of your code over time. An easy way to achieve this is with a static GitHub Pages site and some JavaScript to render the results.

A repository that demonstrates this approach can be found here: martincostello/benchmarks-demo.

Warning

Do not use this action with pull requests that come from untrusted sources, such as from public forks or Dependabot. A malicious user could craft a pull request that hijacks the permissions of the configured GitHub token. See this GitHub Security blog post for more information.

Example Usage

Below is an example of a full GitHub Actions workflow to run the benchmarks in a .NET project and then publish the results to the gh-pages branch of the same GitHub repository.

Important

If pushing to the same repository, ensure that you do not create a circular workflow that triggers itself when the results are pushed to the repository.

name: benchmark

on:
  push:
    branches: [ main ]

permissions:
  contents: read

jobs:
  benchmark:
    runs-on: ubuntu-latest
    permissions:
      contents: write
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-dotnet@v4
    - run: dotnet run --project Benchmarks.csproj --configuration Release
    - uses: martincostello/benchmarkdotnet-results-publisher@v1

If you wish to publish the results to a different repository, you can specify the repository and access token to use as shown in the example below.

Important

GITHUB_TOKEN cannot be used to push results to a different repository. You must use a personal access token or a token for a GitHub App instead.

name: benchmark

on:
  push:
    branches: [ main ]

permissions:
  contents: read

jobs:
  benchmark:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-dotnet@v4
    - run: dotnet run --project Benchmarks.csproj --configuration Release
    - uses: martincostello/benchmarkdotnet-results-publisher@v1
      with:
        repo: '${{ github.repository_owner }}/benchmarks' # Publish to a different repository
        repo-token: ${{ secrets.BENCHMARKS_TOKEN }}       # Use a secret for the access token

Use case

This action is useful for projects that use BenchmarkDotNet to run benchmarks and want to publish the results to a GitHub repository.

For example, I use this action in various repositories of my own, which publish the results of their benchmarks to my benchmarks repository. That data from that repository is then consumed by a GitHub Pages site to display the results using static HTML pages and Plotly.

Features

• Publishes the results of BenchmarkDotNet benchmarks to a Git branch in a GitHub repository.
• Write the Markdown summary of the BenchmarkDotNet results to the workflow run summary.
• Optionally fail the workflow if the duration or memory threshold is exceeded for any benchmark using.
• Optionally posts a comment to a commit or pull request if the duration or memory threshold is exceeded for any benchmark (example).
• Optionally truncate the number of results for each suite of benchmark results to a fixed sliding window.

Tip

This action is only intended to be used to track the relative performance of your benchmarks over time when using GitHub-hosted runners. For more stable results or for tracking absolute performance, consider using a self-hosted runner to run your benchmarks.

Inputs

Name	Description	Default
branch	The optional Git branch to push the results to.	gh-pages
comment-on-threshold	Whether to post a comment if either the duration or memory threshold is exceeded for any benchmark.	false
commit-message	The optional Git commit message to use.	-
commit-message-prefix	The optional prefix to use when generating Git commit messages if a custom commit message is not used.	-
fail-on-threshold	Whether to cause the workflow to fail if either the duration or memory threshold is exceeded for any benchmark.	false
fail-threshold-duration	The optional threshold, as a ratio, which determines if the current workflow fails based on the previous result for a duration metric.	2 (i.e. 200%)
fail-threshold-memory	The optional threshold, as a ratio, which determines if the current workflow fails based on the previous result for a memory usage metric.	2 (i.e. 200%)
max-items	The optional maximum number of results to include for each suite of benchmark results.	Unlimited
name	The optional name to use to group the benchmark results that are found into suites.	Inferred from BenchmarkDotNet results
output-file-path	The optional path of the file to write the results to.	./data.json
output-step-summary	Whether to output the Markdown results from BenchmarkDotNet to GITHUB_STEP_SUMMARY.	true
repo	The optional GitHub repository to push the results to.	github.repository
repo-token	The GitHub access token to use to push the results to a GitHub repository.	github.token
results-path	The optional path of the BenchmarkDotNet results directory to process.	./BenchmarkDotNet.Artifacts

Outputs

None.

Permissions

This action requires the following permissions, depending on the features used:

• contents: write - Required to read and write the results to the target repository.
• issues: write - Required to post comments if comment-on-threshold is enabled.

Feedback

Any feedback or issues can be added to the issues for this project in GitHub.

Repository

The repository is hosted in GitHub: https://github.com/martincostello/benchmarkdotnet-results-publisher.git

License

This project is licensed under the Apache 2.0 license.