Nicholas Meade, Elinor Poole-Dayan, Siva Reddy
This repository contains the official source code for An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models presented at ACL 2022.
For tracking progress on the intrinsic bias benchmarks evaluated in this work, we created Bias Bench. We plan to update Bias Bench in the future with additional bias benchmarks. To make a submission to Bias Bench, please contact nicholas.meade@mila.quebec.
git clone https://github.com/mcgill-nlp/bias-bench.git
cd bias-bench
python -m pip install -e .
Below, a list of the external datasets required by this repository is provided:
Dataset | Download Link | Notes | Download Directory |
---|---|---|---|
Wikipedia-2.5 | Download | English Wikipedia dump used for SentenceDebias and INLP. | data/text |
Wikipedia-10 | Download | English Wikipedia dump used for CDA and Dropout. | data/text |
Each dataset should be downloaded to the specified path, relative to the root directory of the project.
We provide scripts for running all of the experiments presented in the paper.
Generally, each script takes a --model
argument and a --model_name_or_path
argument.
We briefly describe the script(s) for each experiment below:
- CrowS-Pairs: Two scripts are provided for evaluating models against CrowS-Pairs:
experiments/crows.py
evaluates non-debiased models against CrowS-Pairs andexperiments/crows_debias.py
evaluates debiased models against CrowS-Pairs. - INLP Projection Matrix:
experiments/inlp_projection_matrix.py
is used to compute INLP projection matrices. - SEAT: Two scripts are provided for evaluating models against SEAT:
experiments/seat.py
evaluates non-debiased models against SEAT andexperiments/seat_debias.py
evaluates debiased models against SEAT. - StereoSet: Two scripts are provided for evaluating models against StereoSet:
experiments/stereoset.py
evaluates non-debiased models against StereoSet andexperiments/stereoset_debias.py
evaluates debiased models against StereoSet. - SentenceDebias Subspace:
experiments/sentence_debias_subspace.py
is used to compute SentenceDebias subspaces. - GLUE:
experiments/run_glue.py
is used to run the GLUE benchmark. - Perplexity:
experiments/perplexity.py
is used to compute perplexities on WikiText-2.
For a complete list of options for each experiment, run each experiment script with the --h
option.
For example usages of these experiment scripts, refer to batch_jobs
.
The commands used in batch_jobs
produce the results presented in the paper.
- To run SentenceDebias models against any of the benchmarks, you will first need to run
experiments/sentence_debias_subspace.py
. - To run INLP models against any of the benchmarks, you will first need to run
experiments/inlp_projection_matrix.py
. export
contains a collection of scripts to format the results into the tables presented in the paper.
We provide scripts for running all of the experiments presented in the paper on a SLURM cluster in batch_jobs
.
If you plan to use these scripts, make sure you customize python_job.sh
to run the jobs on your cluster.
In addition, you will also need to change both the output (-o
) and error (-e
) paths.
This repository makes use of code from the following repositories:
- Towards Debiasing Sentence Representations
- StereoSet: Measuring Stereotypical Bias in Pre-trained Language Models
- CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Models
- On Measuring Social Biases in Sentence Encoders
- Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection
- Towards Understanding and Mitigating Social Biases in Language Models
- Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP
We thank the authors for making their code publicly available.
If you use the code in this repository, please cite the following paper:
@inproceedings{meade_2022_empirical,
title = "An Empirical Survey of the Effectiveness of Debiasing Techniques for Pre-trained Language Models",
author = "Meade, Nicholas and Poole-Dayan, Elinor and Reddy, Siva",
booktitle = "Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
month = may,
year = "2022",
address = "Dublin, Ireland",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.acl-long.132",
doi = "10.18653/v1/2022.acl-long.132",
pages = "1878--1898",
}