Document a Wasmtime-specific vulnerability runbook

This commit codifies the process [documented
here](https://github.com/bytecodealliance/rfcs/blob/main/accepted/vulnerability-response-runbook.md)
in the Wasmtime repository as it relates to Wasmtime itself. There's
also a few minor changes from recent advisories such as:

* We'll no longer use the publish-the-changes-from-the-advisory feature
  from GitHub. That basically just doesn't work any more.
* PRs will instead be manually created to have CI run on them to weed
  out any issues.
* Details about preparing the `main` branch ahead of the release are
  interleaved with the rest of the runbook.

The intention is to supplement the official runbook with
Wasmtime-specific information and flesh out a few minor steps we're
following that are "extra" here too.

docs/SUMMARY.md

@@ -48,6 +48,7 @@
 - [Security](security.md)
   - [Disclosure Policy](./security-disclosure.md)
   - [What is considered a security bug?](./security-what-is-considered-a-security-vulnerability.md)
+  - [Vulnerability runbook](./security-vulnerability-runbook.md)
 - [Contributing](contributing.md)
   - [Architecture](./contributing-architecture.md)
   - [Building](./contributing-building.md)

docs/contributing-release-process.md

@@ -113,46 +113,8 @@ Like above human interaction is indicated with **bold** text in these steps.
 
 ## Releasing a security patch
 
-When making a patch release that has a security-related fix the contents of the
-patch are often kept private until the day of the patch release which means that
-the process here is slightly different from the patch release process above. In
-addition the precise [runbook is currently under discussion in an
-RFC](https://github.com/bytecodealliance/rfcs/pull/20) for security patches, so
-this intends to document what we've been doing so far and it'll get updated when
-the runbook is merged.
-
-1. **The fix for the security issue is developed in a GitHub Security
-   Advisory**
-   * This will not have any CI run, it's recommended to run `./ci/run-tests.sh`
-     locally at least.
-   * Develop fixes for all branches that will get a patch release in the
-     advisory, one PR per branch. When the advisory is published all branches
-     will be merged simultaneously. Be sure to run `./ci/run-tests.sh` in each
-     branch.
-   * Don't forget to update `RELEASES.md` with notes about the release on
-     each branch.
-2. **Send a PR for the version bump when an email goes out announcing there will
-   be a security release**
-   * An email is sent to the bytecodealliance security mailing list ahead of a
-     patch release to announce that a patch release will happen. At this time you
-     should [trigger the version bump][ci-trigger] against the appropriate
-     `release-x.y.z` branches with the `release-patch` argument.
-   * This will send a PR, but you should not merge it. Instead use this PR and
-     the time ahead of the security release to fix any issues with CI. Older
-     `release-x.y.z` branches haven't run CI in awhile so they may need to
-     backport fixes of one variety or another. DO NOT include the actual fix for
-     the security issue into the PR, that comes in the next step.
-3. **Make the advisories/patches public**
-   * Publishing the GitHub Security Advisory will merge all the PRs into each
-     branch from the advisory. Note that CI will run for release branches but
-     `main` will probably fail CI since it expected to be merged through the
-     merge queue, but that's ok.
-   * Double-check that CI for release branches finishes and completes
-     successfully.
-4. **Merge the version-bump PR**
-   * Like the patch release process this will kick everything else into motion.
-     Note that the actual security fixes should be merged either before or as
-     part of this PR.
+For security releases see the documentation [of the vulnerability
+runbook](./security-vulnerability-runbook.md).
 
 ## Releasing Notes
 

docs/security-vulnerability-runbook.md

@@ -0,0 +1,99 @@
+# Vulnerability Runbook
+
+This document outlines how Wasmtime maintainers should respond to a security
+vulnerability found in Wasmtime. This is intended to be a Wasmtime-specific
+variant of the [runbook
+RFC](https://github.com/bytecodealliance/rfcs/blob/main/accepted/vulnerability-response-runbook.md)
+originally created. More details are available in the RFC in some specific steps.
+
+Vulnerabilities and advisories are all primarily coordinated online through
+GitHub Advisories on the Wasmtime repository. Anyone can make an advisory on
+Wasmtime, and once created anyone can be added to an advisory. Once an advisory
+is created these steps are followed:
+
+1. An **Incident Manager** is selected. By default this is the Wasmtime
+   maintainer that opened the advisory. If a contributor opened the advisory
+   then it's by default the first responder on the advisory. The incident
+   manager can, at any time, explicitly hand off this role to another
+   maintainer.
+
+2. **Fill out the advisory details**. This step involves filling out all the
+   fields on the GitHub Advisory page such as:
+
+   * Description - the description field's initial placeholder has the various
+     sections to fill out. At this point at least a brief description of the
+     impact should be filled out. This will get fleshed out more later too.
+   * Affected versions - determine which previously released versions of
+     Wasmtime are affected by this issue.
+   * Severity - use the CVSS calculator to determine the severity of this
+     vulnerability.
+
+3. **Collaborate on a fix**. This should be done in a private fork created for
+   the security advisory. This is also when any collaborators who can help with
+   the development of the fix should also be invited. At this time only the
+   `main` branch needs to have a fix.
+
+4. **Finalize vulnerability details and patched versions**. After a fix has been
+   developed and the vulnerability is better understood at this point the
+   description of the advisory should be fully filled out and be made ready to
+   go to the public. This is also when the incident manager should determine the
+   number of versions of Wasmtime to patch. The latest two versions are
+   required, and older versions are optional.
+
+5. **Request a CVE**. Use the Big Green Button on the advisory to request a CVE
+   number from GitHub staff.
+
+6. **Send advanced disclosure email**. The incident manager will decide on a
+   disclosure date, typically no more than a week away, and send mail to
+   sec-announce@bytecodealliance.org about the upcoming security release. An
+   example mail [looks like
+   this](https://groups.google.com/a/bytecodealliance.org/g/sec-announce/c/7SjEU_qSE4U/m/Y6baLYkhAgAJ)
+
+7. **Add more stakeholders** (optional). Users interested in getting advanced
+   notice about this vulnerability may respond to the mailing list post. The
+   incident manager will add them to the security advisory.
+
+8. **Prepare PRs for patch releases**. This will involve creating more pull
+   requests in the private fork attached to the advisory. Each version of
+   Wasmtime being patched should have a PR ready-to-go which cleanly applies.
+   Be sure to write release notes on the PR for each release branch.
+
+9. **The full test suite should be run locally for `main`**. Locally try to run
+   as much of the CI matrix as you can. You probably won't be able to run all of
+   it, and that's ok, but try to get the ones that may have common failures.
+   This is required because CI doesn't run on private forks.
+
+10. **Open version bump PRs on the public repository**. Use the
+    [online trigger][ci-trigger] for this workflow to open PRs for all versions
+    that are going to be patched. DO NOT include patch notes or release notes
+    for this fix. Use this time to fix CI by landing PRs to the release branches
+    separate from the version bump PR. DO NOT merge the version bump PR.
+
+11. **Manually make PRs on release day**. DO NOT merge via the security
+    advisory. This has generally not worked well historically because there's
+    too many CI failures and branch protections. On the day of the release make
+    public PRs from all of the previously-created PRs on the private fork.
+    You'll need to push the changes to your own personal repository for this,
+    but that's ok since it's time to make things public anyway. Merge all PRs
+    (including to `main`) once CI passes.
+
+12. **Merge version bump PRs**. Once the fixes have all been merged and CI is
+    green merge all the version bump PRs. That will trigger the automatic
+    release process which will automatically publish to crates.io and publish
+    the release.
+
+13. **Publish the GitHub Advisories**. Delete the private forks and hit that Big
+    Green Button to publish the advisory.
+
+14. **Send mail about the security release**. Send another around of mail to
+    sec-announce@bytecodealliance.org describing the security release. This mail
+    looks [like
+    this](https://groups.google.com/a/bytecodealliance.org/g/sec-announce/c/7SjEU_qSE4U/m/zjW9fWlcAAAJ).
+
+You'll want to pay close attention to CI on release day. There's likely going to
+be CI failures with the fix for the vulnerability for some build configurations
+or platforms and such. It should be easy to fix though so mostly try to stay on
+top of it. Additionally be sure to carefully watch the publish process to
+crates.io. It's possible to hit rate limits in crate publication which
+necessitates a retry of the job later. You can also try publishing locally too
+from the release branch, but it's best to do it through CI.