A Princeton Researcher Proposes Recurrent Looped Transformer (RLT) that Carries Decoder State across Every Token, Fixing 96 Blocks per Token with Unbounded Temporal Depth

Princeton researcher Yifan Zhang has introduced the Recurrent Looped Transformer (RLT), an architecture designed to address memory and context limitations in large language models. The technical report details a causal encoder paired with a recurrent decoder that preserves hidden states across tokens.

Unlike standard transformers that reset state at serving boundaries, RLT carries its final hidden state and a layerwise sliding-window attention cache across every prompt and response token. This design fixes 96 transformer blocks per token while allowing the model to operate with unbounded temporal depth.

By maintaining decoder state continuously, RLT aims to process arbitrarily long sequences without the quadratic memory costs typically associated with extended context windows. This could make long-context inference more efficient and scalable for deployment.

The approach matters because context length and memory bottlenecks remain significant challenges for LLM applications. If validated, recurrent looped architectures could offer a practical alternative to simply scaling up attention, potentially reducing compute requirements for long-form generation and multi-turn dialogue.

This page provides an editorial summary based on publicly available information. It is not a republished article. Use the source link below for the original report.