v0.6.0

Added support for Compact Transformers with:

• Shared-QK, Shared-KV multi-head attention
• Cross-layer sharing

v0.4.0

Main changes:

• Added Radix tokenizer

v0.3.0

Main changes:

• simplify forward method of all models
• remove vocab_size + 1 and change default vocab_size from 10000 to 10001
• cleanup train_*.py
• update Config compat() function
• refactor cached transformer base code
• update transformer argparse argument names
• remove --tokenizer_vocab_size argument
• cleanup

v0.2.0

added:

• SCST sampling with beam search
• SCST baseline using greedy search or sampled captions
• config file now has version
• removed dependency on tensorboardX