MoBA: Mixture of Bi-directional Adapter for Multi-modal Sarcasm Detection

Overview

This repository contains the offical implementation of paper:

MoBA: Mixture of Bi-directional Adapter for Multi-modal Sarcasm Detection, ACM MM 2024

Experiments

1. Install requirements.

   pip install -r requirements.txt
   

2. Download the data.

   Please visit here to download the image data and put them into folder data/dataset_image.

3. Download pre-train models.

   Please donwload the CLIP locally.

4. Run the scripts.

   # MMSD2.0
   sh scripts/2train.sh
   
   # MMSD
   sh scripts/train.sh
   

Reference

If you find this project useful for your research, please consider citing the following paper.

@inproceedings{xie2024moba,
  title={MoBA: Mixture of Bi-directional Adapter for Multi-modal Sarcasm Detection},
  author={Xie, Yifeng and Zhu, Zhihong and Chen, Xin and Chen, Zhanpeng and Huang, Zhiqi},
  booktitle={Proc. of ACM MM},
  year={2024}
}

Acknowledge

• We sincerely thank the Multi-view CLIP, as most of our code is based on it.
• We also greatly appreciate HKE and DynRT.