Recurrent Looped Transformer

Global Tech Sun, 13 Sep 2026 07:42:13 GMT Moderate confidence — 64/100

Abstract Recurrent Looped Transformer (RLT) combines a causal encoder with a recurrent decoder that carries its final hidden state and layerwise sliding-window attention (SWA) cache across every prompt and response token.
The encoder constructs global key–value memory; the decoder extends a continuous latent computation as the sequence grows.
The design brings together latent reasoning with unbounded temporal depth, model–hardware co-design, and model–RL algorithm co-design.

Abstract Recurrent Looped Transformer (RLT) combines a causal encoder with a recurrent decoder that carries its final hidden state and layerwise sliding-window attention (SWA) cache across every prompt and response token.
The encoder constructs global key–value memory; the decoder extends a continuous latent computation as the sequence grows.
The design brings together latent reasoning with unbounded temporal depth, model–hardware co-design, and model–RL algorithm co-design.

Sources: Github