This is the official repository of the *SEM 2022 paper Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Induction. We investigate the use of pretrained language models (LM) for taxonomy learning in a zero-shot setting using prompting and sentence-scoring methods. Through extensive experiments on public benchmarks from TExEval-1 and TExEval-2, we show that our proposed approaches outperform some supervised methods and are competitive with SOTA under certain conditions.
conda create -n taxonomy -y python=3.7 && conda activate taxonomy
-
Install the required packages
pip install -r requirements.txt
-
Install MXNet based on CUDA version
nvcc --version # to check CUDA version
pip install <mxnet> # corresponding MXNet version
The experiments can be run via a bash script that generates and evaluates taxonomies using a single command.
./run.sh <method_name> <model_checkpoint> <domain> <prompt_type>
Here,
- method_name:
{prompt-mlm, restrict-mlm, lm-scorer}
- model_checkpoint:
{bert-base-uncased, bert-large-uncased, roberta-base, roberta-large}
(gpt2
andgpt2-medium
can also be used for LMScorer) - domain:
{equipment, environment, food, science_ev, science_wn, science}
- prompt_type:
{gen, spec, type}
The taxonomies are generated in the directory output/{method_name}/{model_checkpoint}
and the corresponding results are saved as results/{method_name}.csv
.
Currently, taxonomies with top-k hypernyms for each term are generated where k in {1, 3, 5}.
If our research helps you, please kindly cite our paper:
@inproceedings{jain-espinosa-anke-2022-distilling,
title = "Distilling Hypernymy Relations from Language Models: On the Effectiveness of Zero-Shot Taxonomy Induction",
author = "Jain, Devansh and
Espinosa Anke, Luis",
booktitle = "Proceedings of the 11th Joint Conference on Lexical and Computational Semantics",
month = jul,
year = "2022",
address = "Seattle, Washington",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.starsem-1.13",
doi = "10.18653/v1/2022.starsem-1.13",
pages = "151--156",
abstract = "In this paper, we analyze zero-shot taxonomy learning methods which are based on distilling knowledge from language models via prompting and sentence scoring. We show that, despite their simplicity, these methods outperform some supervised strategies and are competitive with the current state-of-the-art under adequate conditions. We also show that statistical and linguistic properties of prompts dictate downstream performance.",
}
The code is implemented using transformers and mlm-scoring.