A Mathematical Framework for Transformer Circuits
- Three Kinds of Composition Path Expansion of Logits Path Expansion of Attention Scores QK Circuit Analyzing a Two-Layer Model Induction Heads Term Importance Analysis Virtual Attention Heads Tra
Unverified
- Three Kinds of Composition Path Expansion of Logits Path Expansion of Attention Scores QK Circuit Analyzing a Two-Layer Model Induction Heads Term Importance Analysis Virtual Attention Heads Tra
Sources: Transformer-circuits