Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added recipe unifying script #1560

Merged
merged 20 commits into from
Oct 31, 2023
Merged
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
7f26ddc
working script + docs
shaydeci Oct 23, 2023
d08558e
renamed script
shaydeci Oct 23, 2023
0463b0c
added tests
shaydeci Oct 23, 2023
f663300
added tests to suite
shaydeci Oct 23, 2023
3ea4d29
Merge branch 'master' into feature/SG-1179_unify_recipe_script
shaydeci Oct 23, 2023
eaadf73
Merge branch 'master' into feature/SG-1179_unify_recipe_script
shaydeci Oct 23, 2023
2095ed5
Merge branch 'master' into feature/SG-1179_unify_recipe_script
shaydeci Oct 24, 2023
742ede0
Merge branch 'master' into feature/SG-1179_unify_recipe_script
BloodAxe Oct 24, 2023
5fa8827
save path resolved in main
shaydeci Oct 29, 2023
e4c9daa
Merge remote-tracking branch 'origin/feature/SG-1179_unify_recipe_scr…
shaydeci Oct 29, 2023
f791e29
used logger for script print
shaydeci Oct 29, 2023
6ad2348
updated positional args
shaydeci Oct 29, 2023
9211aac
Merge remote-tracking branch 'origin/master' into feature/SG-1179_uni…
shaydeci Oct 29, 2023
6ea5ff2
updated test
shaydeci Oct 29, 2023
6276b08
Merge remote-tracking branch 'origin/master' into feature/SG-1179_uni…
shaydeci Oct 30, 2023
df1cbc6
removed redundant cleanup in test
shaydeci Oct 30, 2023
ade92bf
Merge branch 'master' into feature/SG-1179_unify_recipe_script
BloodAxe Oct 30, 2023
a070d3a
Merge branch 'master' into feature/SG-1179_unify_recipe_script
Louis-Dupont Oct 30, 2023
20fcd42
Merge branch 'master' into feature/SG-1179_unify_recipe_script
Louis-Dupont Oct 31, 2023
d8be0cc
Merge branch 'master' into feature/SG-1179_unify_recipe_script
shaydeci Oct 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions src/super_gradients/scripts/merge_recipe.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
"""
Script that saves a merged (i.e no inheritance from other yaml configuration files),
.yaml file that can be ran on its own without the need to keep other configurations which the original
file inherits from.

Usage:
python merge_recipe --config_name=cifar10_resnet -> saves cifar10_resnet_complete.yaml in path/to/super_gradients/recipes
shaydeci marked this conversation as resolved.
Show resolved Hide resolved

python merge_recipe --config_dir=/path/to/recipes/ config_name=my_recipe.yaml -> saves config_name_complete.yaml in /path/to/recipes/

python merge_recipe --config_dir=/path/to/recipes/ config_name=my_recipe.yaml save_path=/other/recipes/dir/my_complete_recipe.yaml
-> saves the complete receip in /other/recipes/dir/my_complete_recipe.yaml

:arg config_name: The .yaml config filename (can leave the .yaml postfix out, but not mandatory).

:arg config_dir: The config directory path, as absolute file system path.
When None, will use SG's recipe directory (i.e path/to/super_gradients/recipes)

:arg: The output path for the complete .yaml file.
When None, will use config_dir/config_name with a trailing "_complete.yaml".
"""

import argparse
from hydra import initialize_config_dir, compose
from hydra.core.global_hydra import GlobalHydra
import pkg_resources
from omegaconf import OmegaConf
import os
from super_gradients import init_trainer
from super_gradients.common.environment.cfg_utils import normalize_path

if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--config_dir", type=str, default=pkg_resources.resource_filename("super_gradients.recipes", ""), help="The config directory path")
parser.add_argument("--config_name", type=str, help=".yaml filename")
parser.add_argument("--save_path", type=str, default=None, help="Destination path to the output .yaml file")
args = parser.parse_args()
if args.save_path is None:
args.save_path = os.path.join(args.config_dir, args.config_name).replace(".yaml", "") + "_complete.yaml"
shaydeci marked this conversation as resolved.
Show resolved Hide resolved

# NEED TO REGISTER RESOLVERS FIRST
init_trainer()
shaydeci marked this conversation as resolved.
Show resolved Hide resolved
GlobalHydra.instance().clear()
with initialize_config_dir(config_dir=normalize_path(args.config_dir), version_base="1.2"):
cfg = compose(config_name=args.config_name)
OmegaConf.save(config=cfg, f=args.save_path)
print(f"Successfully saved recipe at {args.save_path}")