Linear-Attention-Mechanism

⭐ Welcome to my HomePage ⭐

This repository implementatesLinear-Attention-Mechanism based on PyTorch.

The detailed formula can be seen in the Linear Attention Mechanism: An Efficient Attention for Semantic Segmentation or Multi-stage Attention ResU-Net for Semantic Segmentation of Fine-Resolution Remote Sensing Images.

If our code is helpful to you, please cite

1. Li R, Jianlin Su, Duan C and Zheng S. Linear Attention Mechanism: An Efficient Attention for Semantic Segmentation[J]. arXiv preprint arXiv:2007.14902, 2020.
2. 苏剑林. (2020, Jul 04). 《线性Attention的探索：Attention必须有个Softmax吗？ 》[Blog post]. Retrieved from https://spaces.ac.cn/archives/7546
3. R. Li, S. Zheng, C. Duan, J. Su and C. Zhang. "Multistage Attention ResU-Net for Semantic Segmentation of Fine-Resolution Remote Sensing Images." in IEEE Geoscience and Remote Sensing Letters, doi: 10.1109/LGRS.2021.3063381.