Hi, I'm Aleksandrs Koselevs.
- This is a fork of a great pytorch implementation at sainathadapa/attention-primer-pytorch
- Which itself is a fork of a great tutorial at greentfrapp/attention-primer
attention_primer.ipynb
tries to unify the 5 lessons in a single notebook- My notes appear as A: italic
- All training code is removed in the notebook, only doing inference
- You don't need to train anything. However, there might be some references to
training
or--parameter=False
in the descriptions - If you want to train something, the task folders contain the training code
- The implementation in the notebooks might diverge from those in the task folders
- Most images from Vaswani et al. (2017)
- The example code in the descriptions in the notebook might diverge from the one in Implementation
batch_size
defaults to 1, to simplify things- The model implementations are in
all_models.py
, which are copied into the notebooks