This respository contains data and scripts associated to the paper: Human Judgement as a Compass to Navigate Automatic Metrics for Formality Transfer (HumEval 2022) and Multidimensional Evaluation for Text Style Transfer Using ChatGPT. If you use any of the following contents, please cite our paper:
@inproceedings{lai-etal-2022-human,
title = "Human Judgement as a Compass to Navigate Automatic Metrics for Formality Transfer",
author = "Lai, Huiyuan and
Mao, Jiali and
Toral, Antonio and
Nissim, Malvina",
booktitle = "Proceedings of the 2nd Workshop on Human Evaluation of NLP Systems (HumEval)",
month = may,
year = "2022",
address = "Dublin, Ireland",
publisher = "Association for Computational Linguistics",
url = "https://aclanthology.org/2022.humeval-1.9",
pages = "102--115"
}
@misc{lai2023multidimensional,
title={Multidimensional Evaluation for Text Style Transfer Using ChatGPT},
author={Huiyuan Lai and Antonio Toral and Malvina Nissim},
year={2023},
eprint={2304.13462},
archivePrefix={arXiv},
primaryClass={cs.CL}
}
Note: All data used in these two papers are placed in the chatgpt in json format. The following are Automatic Evaluation and Text Style Transfer Papers published
over the last three years in the ACL Anthology.
# Train Classifier/Regressor
python train_style.py -dataset xformal -task single_label_classification/regression
# Evaluation
python eval_style.py -dataset xformal -model BART -task single_label_classification/regressionpython eval_content.py data/output/BART.human.txt data/output/BART.auto.txt# Train GPT2 based Language Model
python train_lm.py -dataset xformal -style 0
python train_lm.py -dataset xformal -style 1
# Evaluation
python eval_fluency.py -model BART Note: All the models, including style regressor, classifiers and gpt-2 for fluency, can be found here.
- Thank you BART! Rewarding Pre-Trained Models Improves Formality Style Transfer. ACL 2021. [code]
- Enhancing Content Preservation in Text Style Transfer Using Reverse Attention and Conditional Layer Normalization. ACL 2021. [code]
- Improving Formality Style Transfer with Context-Aware Rule Injection. ACL 2021.
- Style is NOT a single variable: Case Studies for Cross-Stylistic Language Understanding. ACL 2021. [code]
- TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling. ACL 2021.
- Counterfactuals to Control Latent Disentangled Text Representations for Style Transfer. ACL 2021.
- NAST: A Non-Autoregressive Generator with Word Alignment for Unsupervised Text Style Transfer. ACL Findings 2021. [code]
- LEWIS: Levenshtein Editing for Unsupervised Text Style Transfer. ACL Findings 2021. [code]
- Unsupervised Aspect-Level Sentiment Controllable Style Transfer. AACL 2021.
- Text Style Transfer: Leveraging a Style Classifier on Entangled Latent Representations. RepL4NLP-2021.
- Multi-Pair Text Style Transfer for Unbalanced Data via Task-Adaptive Meta-Learning. METANLP 2021.
- Transductive Learning for Unsupervised Text Style Transfer. EMNLP 2021. [code]
- Generic resources are what you need: Style transfer tasks without task-specific parallel training data. EMNLP 2021. [code]
- Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer. EMNLP 2021. [code]
- Collaborative Learning of Bidirectional Decoders for Unsupervised Text Style Transfer. EMNLP 2021. [code]
- Exploring Non-Autoregressive Text Style Transfer. EMNLP 2021. [code]
- Rethinking Sentiment Style Transfer. EMNLP Findings 2021.
- StylePTB: A Compositional Benchmark for Fine-grained Controllable Text Style Transfer. NAACL 2021. [code]
- Olá, Bonjour, Salve! XFORMAL: A Benchmark for Multilingual Formality Style Transfer. NAACL 2021. [code]
- Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus. NAACL 2021.
- On Learning Text Style Transfer with Direct Rewards. NAACL 2021. [code]
- Civil Rephrases Of Toxic Texts With Self-Supervised Transformers. EACL 2021. [code]
- Expertise Style Transfer: A New Task Towards Better Communication between Experts and Laymen. ACL 2020. [code]
- Parallel Data Augmentation for Formality Style Transfer. ACL 2020. [code]
- Exploring Contextual Word-level Style Relevance for Unsupervised Style Transfer. ACL 2020. [code]
- Politeness Transfer: A Tag and Generate Approach. ACL 2020. [code]
- Learning to Generate Multiple Style Transfer Outputs for an Input Sentence. NGT 2020.
- Challenges in Emotion Style Transfer: An Exploration with a Lexical Substitution Pipeline. SOCIALNLP 2020. [code]
- Reformulating Unsupervised Style Transfer as Paraphrase Generation. EMNLP 2020. [code]
- Generating similes effortlessly like a Pro: A Style Transfer Approach for Simile Generation. EMNLP 2020. [code]
- DGST: a Dual-Generator Network for Text Style Transfer. EMNLP 2020. [code]
- Unsupervised Text Style Transfer with Padded Masked Language Models. EMNLP 2020.
- StyleDGPT: Stylized Response Generation with Pre-trained Language Models. EMNLP Findings 2020. [code]
- Semi-supervised Formality Style Transfer using Language Model Discriminator and Mutual Information Maximization. EMNLP Findings 2020. [code]
- Contextual Text Style Transfer. EMNLP Findings 2020.
- Disentangled Representation Learning for Non-Parallel Text Style Transfer. ACL 2019. [code]
- A Hierarchical Reinforced Sequence Operation Method for Unsupervised Text Style Transfer. ACL 2019. [code]
- Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation. ACL 2019. [code]
- “Transforming” Delete, Retrieve, Generate Approach for Controlled Text Style Transfer. EMNLP 2019. [code]
- Domain Adaptive Text Style Transfer. EMNLP 2019. [code]
- Harnessing Pre-Trained Neural Networks with Rules for Formality Style Transfer. EMNLP 2019. [code]
- Multiple Text Style Transfer by using Word-level Conditional Generative Adversarial Network with Two-Phase Training. EMNLP 2019.
- Semi-supervised Text Style Transfer: Cross Projection in Latent Space. EMNLP 2019.
- Evaluating Style Transfer for Text. NAACL 2019. [code]
- Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus. NAACL 2019.