AV-SUPERB

Paper, Submission leaderboard

Model Evaluation

1. Using our toolkit:

Training:

python run_downsteam -m train -u <upstream model name> -d <downstream task name> -s <feature type>

TBA

2. Using our submission platform:

TBA

Citation

@article{tseng2023avsuperb,
  title={AV-SUPERB: A Multi-Task Evaluation Benchmark for Audio-Visual Representation Models},
  author={Yuan Tseng and Layne Berry and Yi-Ting Chen and I-Hsiang Chiu and Hsuan-Hao Lin and Max Liu and Puyuan Peng and Yi-Jen Shih and Hung-Yu Wang and Haibin Wu and Po-Yao Huang and Chun-Mao Lai and Shang-Wen Li and David Harwath and Yu Tsao and Shinji Watanabe and Abdelrahman Mohamed and Chi-Luen Feng and Hung-yi Lee},
  journal={arXiv preprint arXiv:2309.10787},
  year={2023}
}

License

AV-SUPERB is primarily distributed under the terms of both the MIT license and the Apache License (Version 2.0).

Using files and pretrained AV-HuBERT models under the upstream_models/vhubert folder requires accepting the terms in the AV-HuBERT license agreement listed in this file.

See LICENSE-APACHE, LICENSE-MIT, COPYRIGHT for details.

Acknowledgement

Source code is based on S3PRL.