This notebook demonstrates the linear property of transformers from the paper Attention is all you need [https://arxiv.org/abs/1706.03762]. The code shows how positoinal encodings can be represented by linear transformaitons of each other inside the transformer architecture.
-
Notifications
You must be signed in to change notification settings - Fork 0
Demonstrates the property of transformers' positional encodings to be represented by linear function independent on the word indexes
License
malashinroman/pe_computation
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Folders and files
| Name | Name | Last commit message | Last commit date | |
|---|---|---|---|---|
Repository files navigation
About
Demonstrates the property of transformers' positional encodings to be represented by linear function independent on the word indexes
Resources
License
Stars
Watchers
Forks
Releases
No releases published
Packages 0
No packages published