Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
machine-learning
artificial-intelligence
transformer
attention
attention-is-all-you-need
attention-mechanisms
gpt3
gpt4
chatgpt
context-length
-
Updated
Jan 7, 2024 - Python