Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

update Model Wrapper to work with pytorch detection models #188

Open
noknok00 opened this issue Feb 12, 2022 · 2 comments
Open

update Model Wrapper to work with pytorch detection models #188

noknok00 opened this issue Feb 12, 2022 · 2 comments
Labels
enhancement New feature or request

Comments

@noknok00
Copy link

Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

As explained in Slack, I am trying to use BaaL with a Object Detection Model from pytorch.
Since the Model Wrapper was coded with Classification models in mind, it doesn't work very well with Object Detection Models.
I found pytorch use this Training Loop for their Detection Model, so we can take inspiration from it:
https://github.com/pytorch/vision/blob/main/references/detection/engine.py
function "train_one_epoch".

Describe the solution you'd like
A clear and concise description of what you want to happen.

I believe we only need to update the "train_on_batch" function to work with Detection's models.
perhaps one flag to let the Model Wrapper class know if it need to use a "training loop" for classification or for detection.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Active Learning Dataset I believe has a similar situation of being coded with "classification" in mind, and doesn't play well with a "detection" kind of dataset with bounding boxes and labels, rather than just labels as in "classification".
I worked around by using my own Dataset class and Collate Function.
Not sure if I should submit another Feature Request for this.

Additional context
Add any other context or screenshots about the feature request here.

with these changes I believe we could support Semantic and Instance segmentation too, since their Training loop (and datasets) are similar to Object Detection.

@noknok00 noknok00 added the enhancement New feature or request label Feb 12, 2022
@Dref360
Copy link
Member

Dref360 commented Feb 12, 2022

Hello,

I tried to make this work in this gist, but the predict_on_batch doesn't work yet.

If we run model(data) n times, we would get the following output:

[ [{'boxes': tensor([[...]]), 'scores': tensor([...]), 'labels': tensor([...])}],
 [{'boxes': tensor([[...]]), 'scores': tensor([...]), 'labels': tensor([...])}],
...
 [{'boxes': tensor([[...]]), 'scores': tensor([...]), 'labels': tensor([...])}]
]

How should we stack these to feed to the heuristics? The number of boxes is also potentially variable between inferences.

In our experiments, we computed the variance of the raw output directly from the heads.

@Dref360
Copy link
Member

Dref360 commented Mar 14, 2022

Anything I can help with? Happy to schedule a meeting as needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants