This code is licenced CC-BY 4.0: you may use it for commercial and non-commercial purposes, and you MUST acknowledge the source clearly if you so. Acknowledge this reference for Hyperparameter Power Impact in Transformer Language Model Training:
Mads Guldborg Kjeldgaard Kongsbak, Timmie Mikkel Rantzau Lagermann, Lucas Høyberg Puvis de Chavannes, Leon Derczynski. "Hyperparameter Power Impact in Transformer Language Model Training". 2021. Conference Paper, IT University of Copenhagen.
Bibtex:
@article{HyperparameterPowerImpact,
author={Mads Guldborg Kjeldgaard Kongsbak, Timmie Mikkel Rantzau Lagermann, Lucas Høyberg Puvis~de~Chavannes, Leon Derczynski},
title={Hyperparameter Power Impact in Transformer Language Model Training},
year={2021},
institution={IT University of Copenhagen},
}
For details, see LICENSE.md.