Questions about the experiments on the RSNA pneumonia dataset?

[chaoyi-wu]

Can you show the split of the RSNA pneumonia dataset and the code for calculating the metrics?

[ant0nsc]

Can you add a bit more context? Split of the RSNA dataset where - in the ECCV paper?

@ozan-oktay @Shruthi42 FYI

[chaoyi-wu]

yes, in the eccv paper "Making the Most of Text Semantics to Improve Biomedical Vision–Language Processing", section 4.5, you mentioned that you split the RSNA pneumonia dataset into 6634 for training and 2907 for testing. The number seems different compared with the split csv document released by the LoVT paper. So, you re-split the dataset by yourself? May you release the detailed documents?

[ozan-oktay]

Thanks @WUcccccc for the question. For the experiments provided in section 4.5, we have used the bounding box labels that were released post the Kaggle challenge which also contains the severity labels (see below)
https://www.kaggle.com/competitions/rsna-pneumonia-detection-challenge/discussion/150643

The fine-tuning and evaluation were limited only to this subset of the dataset, which should explain the difference you highlighted above. If you would like to run experiments on a different dataset split, please refer to the models released in https://huggingface.co/microsoft/BiomedVLP-CXR-BERT-specialized

We haven't yet released the mIoU and dice metric evaluation code used in the experiments, but happy to share the unpolished version of the metrics code via email if you are trying to meet a deadline. Thanks.

[chaoyi-wu]

Thanks for your reply. wtzxxxwcy02@163.com This is my email address. It would be a great help for me to share the code with me.