docs: Add FSDP to deepspeed (#9182)

determined-ai · May 14, 2024 · 2092943 · 2092943
1 parent 4f180db
commit 2092943
Showing 1 changed file with 7 additions and 0 deletions.
diff --git a/docs/model-dev-guide/api-guides/apis-howto/deepspeed/deepspeed.rst b/docs/model-dev-guide/api-guides/apis-howto/deepspeed/deepspeed.rst
@@ -305,6 +305,13 @@ interleaving micro batches:
 
 .. _deepspeed-profiler:
 
+Fully Sharded Data Parallelism (FSDP)
+=====================================
+
+To use FSDP, use the PyTorch FSDP package as usual along with the Core API. ``PytorchTrial`` API
+does not support FSDP. To find out more about the PyTorch FSDP package, visit `PyTorch tutorials
+<https://pytorch.org/tutorials/>`__ > Getting Started with Fully Sharded Data Parallel (FSDP).
+
 ***********
  Profiling
 ***********