You Could Have Come Up With Kimi Delta Attention | Doubleword
- A note on notation: this article defaults to bra-ket notation because (in my quantum-inspired opinion) it makes the shapes in this derivation very clear.
- The Math notation switch above rewrites every equation using conventional bold vectors and explicit transposes instead.
- In bra-ket mode, ∣q⟩\lvert q\rangle is a column vector, ⟨k∣\langle k\rvert is a row vector, ⟨k∣q⟩\langle k\rvert q\rangle is a number, and ∣v⟩⟨k∣\lvert v\rangle\langle k\rvert is a matrix.
Unverified
- A note on notation: this article defaults to bra-ket notation because (in my quantum-inspired opinion) it makes the shapes in this derivation very clear.
- The Math notation switch above rewrites every equation using conventional bold vectors and explicit transposes instead.
- In bra-ket mode, ∣q⟩\lvert q\rangle is a column vector, ⟨k∣\langle k\rvert is a row vector, ⟨k∣q⟩\langle k\rvert q\rangle is a number, and ∣v⟩⟨k∣\lvert v\rangle\langle k\rvert is a matrix.
Sources: Doubleword