-
Notifications
You must be signed in to change notification settings - Fork 43
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add SemanticSam3dLogger #643
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I went through this again and found a few more small points. Once this is addressed we can merge this.
model_type="vit_b", | ||
checkpoint_path=None, | ||
): | ||
_, sam = get_sam_model( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can't we just pass the checkpoint here? Then we don't need to duplicate the code below.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, that won't work as we need to pass the weights to the wrapper model (to initialize the adapter blocks)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, I don't fully understand. Let's discuss tomorrow :) .
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay now I see already where your suspicion comes from.
TLDR: I think we might have to merge the get_3d_sam_model
into get_sam_model
for the best possible design (which leads to flexibly loading SAM checkpoints for finetuning, and the 3d-SAM checkpoints for downstream semantic inference).
I have a plan on this. I'll take care of this first thing in the morning. Thanks for spotting.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the current design should work at least for intialization. We can revisit this later to discuss how we do this for actually loading the trained 3d models.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okie, I'll leave it to here for now then. Thanks!
model_type="vit_b", | ||
checkpoint_path=None, | ||
): | ||
_, sam = get_sam_model( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, I don't fully understand. Let's discuss tomorrow :) .
Updates to SAM 3d training --------- Co-authored-by: Constantin Pape <constantin.pape@informatik.uni-goettingen.de>
No description provided.