-
Notifications
You must be signed in to change notification settings - Fork 3.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[WIP] Add structured result output #1989
Closed
Closed
Changes from all commits
Commits
Show all changes
177 commits
Select commit
Hold shift + click to select a range
5193004
added result
williamFalcon c84d9db
added train step return
williamFalcon 76892ff
added train step return
williamFalcon 7d90199
added train step return
williamFalcon 8ca8874
added train step return
williamFalcon 7f4e308
added train step return
williamFalcon 2e8d9c4
added train step return
williamFalcon a1809aa
added train step return
williamFalcon b4277ad
added basic template with new syntax and result object
williamFalcon 5bb86f9
added basic template with new syntax and result object
williamFalcon 9008386
added basic template with new syntax and result object
williamFalcon 2be8ad9
added basic template with new syntax and result object
williamFalcon bb180a3
added basic template with new syntax and result object
williamFalcon c943787
added basic template with new syntax and result object
williamFalcon 725477b
added basic template with new syntax and result object
williamFalcon a59fe36
added basic template with new syntax and result object
williamFalcon f8eaceb
added basic template with new syntax and result object
williamFalcon 20c1814
added basic template with new syntax and result object
williamFalcon eacee5d
added basic template with new syntax and result object
williamFalcon c8b5d21
added basic template with new syntax and result object
williamFalcon b27150e
added basic template with new syntax and result object
williamFalcon 10b8860
added basic template with new syntax and result object
williamFalcon 91d3d15
added basic template with new syntax and result object
williamFalcon 186881f
added basic template with new syntax and result object
williamFalcon df5f2c0
added basic template with new syntax and result object
williamFalcon bd97c9d
added basic template with new syntax and result object
williamFalcon c7b3abc
added basic template with new syntax and result object
williamFalcon a40201c
added basic template with new syntax and result object
williamFalcon ff0f582
added basic template with new syntax and result object
williamFalcon a90439f
added basic template with new syntax and result object
williamFalcon f4e94e5
added basic template with new syntax and result object
williamFalcon 70fc2f5
added basic template with new syntax and result object
williamFalcon 60f572b
added basic template with new syntax and result object
williamFalcon 6e66866
added basic template with new syntax and result object
williamFalcon 3c66051
removed args thing
williamFalcon 500be23
removed args thing
williamFalcon 665000f
removed args thing
williamFalcon cf1de29
removed args thing
williamFalcon c5ab427
removed args thing
williamFalcon 8efd923
removed args thing
williamFalcon 3c0cef1
process results
williamFalcon 458afb4
docs
williamFalcon ebbe040
docs
williamFalcon 9b56d44
docs
williamFalcon 3559d58
docs
williamFalcon 30eaa2c
docs
williamFalcon a362b3b
docs
williamFalcon 476837c
docs
williamFalcon 9fde8ef
docs
williamFalcon 5248b5f
docs
williamFalcon 68a0e0e
docs
williamFalcon b3cc286
docs
williamFalcon 928b842
docs
williamFalcon b9ba7df
docs
williamFalcon 4833e9a
docs
williamFalcon f8d0b91
docs
williamFalcon 06980a2
docs
williamFalcon 399b202
docs
williamFalcon b3a91c8
docs
williamFalcon fbe35de
docs
williamFalcon 9e3af3a
docs
williamFalcon 18d5c23
docs
williamFalcon fd20e1a
docs
williamFalcon 858b5ac
docs
williamFalcon bc7b165
docs
williamFalcon ec633ac
docs
williamFalcon 00954c3
docs
williamFalcon 8293740
docs
williamFalcon 11ac6fd
docs
williamFalcon 6baa0a5
docs
williamFalcon 6bfb2b4
docs
williamFalcon 7e434db
docs
williamFalcon a348e5a
docs
williamFalcon 5e99904
docs
williamFalcon cccee53
docs
williamFalcon 8e6c879
docs
williamFalcon ca5254a
docs
williamFalcon 60e265c
docs
williamFalcon e31e088
docs
williamFalcon 99a3457
docs
williamFalcon 16d82a0
docs
williamFalcon 772b40f
docs
williamFalcon 91fc9d4
docs
williamFalcon 5acd39e
docs
williamFalcon 4a1acd1
docs
williamFalcon eb6a36b
docs
williamFalcon 8500551
docs
williamFalcon bcad312
docs
williamFalcon 5cb9026
docs
williamFalcon b07bb94
docs
williamFalcon 7028cc6
docs
williamFalcon f3913ec
docs
williamFalcon 3d38cec
docs
williamFalcon da5fdaa
docs
williamFalcon b8a575b
docs
williamFalcon 7179a0a
docs
williamFalcon 2290201
docs
williamFalcon 4c74043
docs
williamFalcon befe165
docs
williamFalcon 2bb95df
docs
williamFalcon fd61a0a
docs
williamFalcon 90c9ee7
docs
williamFalcon 87500fe
docs
williamFalcon 828c519
docs
williamFalcon 0c68825
docs
williamFalcon 61e82ef
docs
williamFalcon ce666a8
docs
williamFalcon 48404da
docs
williamFalcon c743ff4
docs
williamFalcon 26d6694
docs
williamFalcon 0dfd265
docs
williamFalcon db47ada
docs
williamFalcon ff37d97
docs
williamFalcon 8826b03
docs
williamFalcon 4f32e04
docs
williamFalcon 7a948c4
docs
williamFalcon 61b6bf0
docs
williamFalcon 89c0bfb
docs
williamFalcon 1847612
docs
williamFalcon f644a2c
docs
williamFalcon feabb64
docs
williamFalcon 5379bdb
docs
williamFalcon b18dc0b
docs
williamFalcon 569236d
docs
williamFalcon 4ae3943
docs
williamFalcon 76494d7
docs
williamFalcon e0fcc1d
docs
williamFalcon 34c282e
docs
williamFalcon 6d00501
docs
williamFalcon 6b96497
docs
williamFalcon 1da5abd
docs
williamFalcon 8690f34
docs
williamFalcon a9f2967
tests
williamFalcon 02dbd20
tests
williamFalcon b0c65e1
tests
williamFalcon 6452e49
tests
williamFalcon 1ee2c1f
tests
williamFalcon 4038313
tests
williamFalcon 4404faa
tests
williamFalcon ddeb0b2
tests
williamFalcon b7ed457
tests
williamFalcon 77cafa4
tests
williamFalcon 39c742b
tests
williamFalcon f177dcd
added hparams test
williamFalcon 265ee64
added hparams test
williamFalcon 507eaeb
added hparams test
williamFalcon b2e4664
docs
williamFalcon b1a53e5
docs
williamFalcon a344f98
docs
williamFalcon d4f0e2f
docs
williamFalcon 7221f23
docs
williamFalcon e6da6b0
docs
williamFalcon 9f94f4f
docs
williamFalcon f17d40c
docs
williamFalcon 09f38ad
eval step
williamFalcon 8558165
eval step
williamFalcon 12877bc
eval step
williamFalcon d4c56f7
eval step
williamFalcon 6a52e8d
eval step
williamFalcon 681041e
eval step
williamFalcon 34ace25
eval step
williamFalcon 5855753
eval step
williamFalcon 136cfa0
eval step
williamFalcon b98b29b
eval step
williamFalcon 057d256
eval step
williamFalcon d639742
eval step
williamFalcon 7065e8a
eval step
williamFalcon 869b6e8
eval step
williamFalcon f327c7a
training batch clean up
williamFalcon fe02cbe
training batch clean up
williamFalcon 0fc6e9c
training batch clean up
williamFalcon 3df9b53
training batch clean up
williamFalcon 332c720
training batch clean up
williamFalcon ef3d835
training batch clean up
williamFalcon 491b3f5
training batch clean up
williamFalcon 098a731
training batch clean up
williamFalcon 5b0a805
training batch clean up
williamFalcon File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,147 @@ | ||
""" | ||
Example template for defining a system. | ||
""" | ||
import os | ||
|
||
import torch | ||
from torch import Tensor | ||
import torch.nn as nn | ||
import torch.nn.functional as F | ||
from torch import optim | ||
import torchvision.transforms as transforms | ||
from torchvision.datasets import MNIST | ||
from torch.utils.data import DataLoader | ||
import pytorch_lightning as pl | ||
|
||
|
||
class SuperLitModel(pl.LightningModule): | ||
""" | ||
Sample model to show how to define a template. | ||
|
||
Example: | ||
""" | ||
|
||
def __init__(self, | ||
drop_prob: float = 0.2, | ||
batch_size: int = 2, | ||
in_features: int = 28 * 28, | ||
learning_rate: float = 0.001 * 8, | ||
optimizer_name: str = 'adam', | ||
out_features: int = 10, | ||
hidden_dim: int = 1000, | ||
**kwargs | ||
): | ||
# init superclass | ||
super().__init__() | ||
self.drop_prob = drop_prob | ||
self.batch_size = batch_size | ||
self.in_features = in_features | ||
self.learning_rate = learning_rate | ||
self.optimizer_name = optimizer_name | ||
self.out_features = out_features | ||
self.hidden_dim = hidden_dim | ||
|
||
self.c_d1 = nn.Linear(in_features=self.in_features, | ||
out_features=self.hidden_dim) | ||
self.c_d1_bn = nn.BatchNorm1d(self.hidden_dim) | ||
self.c_d1_drop = nn.Dropout(self.drop_prob) | ||
|
||
self.c_d2 = nn.Linear(in_features=self.hidden_dim, | ||
out_features=self.out_features) | ||
|
||
def forward(self, x): | ||
""" | ||
No special modification required for Lightning, define it as you normally would | ||
in the `nn.Module` in vanilla PyTorch. | ||
""" | ||
x = self.c_d1(x.view(x.size(0), -1)) | ||
x = torch.tanh(x) | ||
x = self.c_d1_bn(x) | ||
x = self.c_d1_drop(x) | ||
x = self.c_d2(x) | ||
return x | ||
|
||
def training_step(self, batch: Tensor, batch_idx: int): | ||
""" | ||
Lightning calls this inside the training loop with the data from the training dataloader | ||
passed in as `batch`. | ||
""" | ||
# forward pass | ||
x, y = batch | ||
y_hat = self(x) | ||
loss = F.cross_entropy(y_hat, y) | ||
|
||
# structure the return from the training loop | ||
step_result = pl.Result( | ||
minimize=loss, | ||
checkpoint_on=loss, | ||
early_stop_on=loss, | ||
) | ||
|
||
step_result.log_metric('train_loss', loss) | ||
step_result.pbar_metric('pbar_loss', loss) | ||
|
||
# return loss | ||
# return step_result | ||
return {'loss': loss, 'log':{'something': 1}, 'random': 'af'} | ||
|
||
def validation_step(self, batch: Tensor, batch_idx: int): | ||
# forward pass | ||
x, y = batch | ||
y_hat = self(x) | ||
val_loss = F.cross_entropy(y_hat, y) | ||
|
||
result = pl.Result() | ||
result.log_metric('val_loss', val_loss) | ||
result.pbar_metric('pbar_loss', val_loss) | ||
|
||
return {'val_loss': val_loss, 'log': {'aa': val_loss}, 'progress_bar': {'aa': val_loss}} | ||
|
||
def configure_optimizers(self): | ||
""" | ||
Return whatever optimizers and learning rate schedulers you want here. | ||
At least one optimizer is required. | ||
""" | ||
return optim.Adam(self.parameters(), lr=self.learning_rate) | ||
|
||
@staticmethod | ||
def add_model_specific_args(parent_parser): # pragma: no-cover | ||
""" | ||
Define parameters that only apply to this model | ||
""" | ||
parser = ArgumentParser(parents=[parent_parser], add_help=False) | ||
parser.add_argument('--in_features', default=28 * 28, type=int) | ||
parser.add_argument('--out_features', default=10, type=int) | ||
parser.add_argument('--hidden_dim', default=5000, type=int) | ||
parser.add_argument('--drop_prob', default=0.2, type=float) | ||
parser.add_argument('--learning_rate', default=0.001, type=float) | ||
parser.add_argument('--data_dir', default='.', type=str) | ||
parser.add_argument('--batch_size', default=64, type=int) | ||
return parser | ||
|
||
|
||
if __name__ == '__main__': | ||
from argparse import ArgumentParser | ||
import pytorch_lightning as pl | ||
|
||
# add trainer args | ||
parser = ArgumentParser() | ||
parser = pl.Trainer.add_argparse_args(parser) | ||
|
||
# add model args | ||
parser = SuperLitModel.add_model_specific_args(parser) | ||
args = parser.parse_args() | ||
|
||
# init data, model | ||
mnist_train = MNIST(args.data_dir, train=True, download=True, transform=transforms.ToTensor()) | ||
mnist_train = DataLoader(mnist_train, batch_size=args.batch_size, num_workers=0) | ||
model = SuperLitModel(**vars(args)) | ||
|
||
# init trainer | ||
trainer = pl.Trainer.from_argparse_args(args) | ||
|
||
trainer.fit( | ||
model, | ||
train_dataloader=mnist_train, | ||
val_dataloaders=mnist_train | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this can be parsed automatically from Models units arguments...