LLM Attention Visualization
- One interesting thing about transformer-based large language models are that, during the generation phase, it is able to draw information from any of its previous tokens.
- But it needs to be selective; if every token affects the generation equally, it won't be very effective.
- This process needs a mechanism to decide how much a token affects the next token.
Unverified
- One interesting thing about transformer-based large language models are that, during the generation phase, it is able to draw information from any of its previous tokens.
- But it needs to be selective; if every token affects the generation equally, it won't be very effective.
- This process needs a mechanism to decide how much a token affects the next token.
Sources: Ishamf