diff --git a/CHANGELOG.md b/CHANGELOG.md
index 336669d767e..9b9e7bad2a4 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -56,6 +56,8 @@ release.
 
 - Clarify that units should use UCUM case sensitive variant.
   ([#3306](https://github.com/open-telemetry/opentelemetry-specification/pull/3306))
+- Recommended non-prefixed units for metric instrument semantic conventions.
+  ([#3312](https://github.com/open-telemetry/opentelemetry-specification/pull/3312))
 - Remove No-Op instrument and Meter creation requirements.
   ([#3322](https://github.com/open-telemetry/opentelemetry-specification/pull/3322))
 - Fixed attributes requirement level in semantic conventions for hardware metrics
diff --git a/specification/metrics/semantic_conventions/README.md b/specification/metrics/semantic_conventions/README.md
index a7d5db37483..ccd9f9e612e 100644
--- a/specification/metrics/semantic_conventions/README.md
+++ b/specification/metrics/semantic_conventions/README.md
@@ -190,6 +190,8 @@ give additional meaning *without* the leading default unit (`1`). For example,
 use `{packet}`, `{error}`, `{fault}`, etc.
 - Instrument units SHOULD be specified using the UCUM case sensitive ("c/s")
   variant. For example, "Cel" for the unit with full name "degree Celsius".
+- Instruments SHOULD use non-prefixed units (i.e. `By` instead of `MiBy`)
+  unless there is good technical reason to not do so.
 
 ### Instrument Types