How to import a fine-tunable detectron 2 model? #5266
Answered
by
david-wei-01001
davidatuken
asked this question in
Q&A
-
I want to use model_zoo.build_model(cfg), but how can I also be able to backdrop on it? |
Beta Was this translation helpful? Give feedback.
Answered by
david-wei-01001
Apr 23, 2024
Replies: 1 comment
-
Here is an expample with X101-FPN: def fasterrcnn_dectron2(
model_str,
model_weight_str=None,
pretrained=True,
num_classes=91,
trainable_backbone_layers=0
):
"""
Constructs a Faster R-CNN model with an X101-FPN backbone from the Detectron2 model zoo.
The input to the model is expected to be a list of tensors, each of shape [C, H, W], one for each
image, and should be in the range [0, 1]. Different images can have different sizes.
The behavior of the model changes depending if it is in training or evaluation mode.
During training, the model expects both the input tensors, as well as targets (list of dictionary),
containing:
- "instances" (Instances): a `Instances` object representing the ground-truth instances in the image.
It should contain fields "gt_boxes" (a `Boxes` object) and "gt_classes" (a `Tensor`).
The model returns a `Dict[str, torch.Tensor]` during training, containing the classification and regression
losses for both the RPN and the R-CNN.
During inference, the model requires only the input tensors, and returns the post-processed
predictions as a `List[Dict[str, torch.Tensor]]`, one for each input image. The fields of the dictionary are as
follows:
- "instances" (Instances): a `Instances` object representing the predicted instances in the image.
It contains fields "pred_boxes" (a `Boxes` object), "scores" (a `Tensor`), and "pred_classes" (a `Tensor`).
Args:
model_str: The string indicating the config file of the pretrained model in detectron2 model_zoo
model_weight_str: The string indicating the pretrained weights of the pretrained model in detectron2 model_zoo
pretrained (bool): If True, returns a model pre-trained on COCO train2017
num_classes (int): number of output classes of the model (including the background)
trainable_backbone_layers (int): number of trainable (not frozen) resnet layers starting from final block.
Valid values are between 0 and 5, with 5 meaning all backbone layers are trainable.
"""
# Create a Detectron2 configuration
cfg = get_cfg()
# Load the X101-FPN model from the Detectron2 model zoo
cfg.merge_from_file(model_zoo.get_config_file(model_str))
# Optionally, you can set threshold for the model
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.5
# Set other parameters
cfg.MODEL.ROI_HEADS.NUM_CLASSES = num_classes
if pretrained:
# Optionally, you can load pre-trained weights
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url(model_weight_str)
# In fine tuning, if you want to leave only the last several layers to be backpropable and all others frozen
# Set the layers to be frozen (C1, C2, C3, ..., Cx where x = 4 - trainable_layers)
to_freeze = [f"res{i}" for i in range(2, 6 - trainable_backbone_layers)]
model = build_model(cfg)
for name, param in model.named_parameters():
for string in to_freeze:
if string in name:
param.requires_grad = False
return model
else:
# Set model weights to None for random initialization
cfg.MODEL.WEIGHTS = None
return build_model(cfg) |
Beta Was this translation helpful? Give feedback.
0 replies
Answer selected by
davidatuken
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Here is an expample with X101-FPN: