forked from Lightning-AI/pytorch-lightning
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
simplify examples structure (Lightning-AI#1247)
* simplify examples structure * update changelog * fix imports * rename example * rename scripts * changelog
- Loading branch information
Showing
20 changed files
with
138 additions
and
84 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,14 +1,67 @@ | ||
# Examples | ||
This folder has 4 sections: | ||
This folder has 3 sections: | ||
|
||
### Basic examples | ||
These show the most common use of Lightning for either CPU or GPU training. | ||
## Basic Examples | ||
Use these examples to test how lightning works. | ||
|
||
### Domain templates | ||
These are templates to show common approaches such as GANs and RL. | ||
#### Test on CPU | ||
```bash | ||
python cpu_template.py | ||
``` | ||
|
||
### Full examples | ||
Contains examples demonstrating ImageNet training, Semantic Segmentation, etc. | ||
--- | ||
#### Train on a single GPU | ||
```bash | ||
python gpu_template.py --gpus 1 | ||
``` | ||
|
||
### Multi-node examples | ||
These show how to run jobs on a GPU cluster using lightning. | ||
--- | ||
#### DataParallel (dp) | ||
Train on multiple GPUs using DataParallel. | ||
|
||
```bash | ||
python gpu_template.py --gpus 2 --distributed_backend dp | ||
``` | ||
|
||
--- | ||
#### DistributedDataParallel (ddp) | ||
|
||
Train on multiple GPUs using DistributedDataParallel | ||
```bash | ||
python gpu_template.py --gpus 2 --distributed_backend ddp | ||
``` | ||
|
||
--- | ||
#### DistributedDataParallel+DP (ddp2) | ||
|
||
Train on multiple GPUs using DistributedDataParallel + dataparallel. | ||
On a single node, uses all GPUs for 1 model. Then shares gradient information | ||
across nodes. | ||
```bash | ||
python gpu_template.py --gpus 2 --distributed_backend ddp2 | ||
``` | ||
|
||
## Multi-node example | ||
|
||
This demo launches a job using 2 GPUs on 2 different nodes (4 GPUs total). | ||
To run this demo do the following: | ||
|
||
1. Log into the jumphost node of your SLURM-managed cluster. | ||
2. Create a conda environment with Lightning and a GPU PyTorch version. | ||
3. Choose a script to submit | ||
|
||
### DDP | ||
Submit this job to run with DistributedDataParallel (2 nodes, 2 gpus each) | ||
```bash | ||
sbatch ddp_job_submit.sh YourEnv | ||
``` | ||
|
||
### DDP2 | ||
Submit this job to run with a different implementation of DistributedDataParallel. | ||
In this version, each node acts like DataParallel but syncs across nodes like DDP. | ||
```bash | ||
sbatch ddp2_job_submit.sh YourEnv | ||
``` | ||
|
||
## Domain templates | ||
These are templates to show common approaches such as GANs and RL. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
File renamed without changes.
2 changes: 1 addition & 1 deletion
2
pl_examples/domain_templates/gan.py → ...n_templates/generative_adversarial_net.py
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Empty file.
44 changes: 0 additions & 44 deletions
44
pl_examples/full_examples/semantic_segmentation/models/unet/model.py
This file was deleted.
Oops, something went wrong.
File renamed without changes.
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
Empty file.