As tasks stretch over hundreds or thousands of steps, language models struggle to keep the full history in memory. Researchers at the Berkeley AI Research Lab have released ABBEL, a system that replaces raw interaction logs with natural-language belief states to maintain performance without consuming excessive context.
The problem with self-summarisation
When models cannot store every previous message, they must summarise the conversation to fit the context window. Tools like Cursor‘s composer 2.5 and Grandcode use this technique, known as context compaction, to manage long interactions.
Despite apparent success on benchmarks, model providers advise against using compaction during active coding sessions. The gap between summarised and full-context performance remains stubborn. In a test involving the Combination Lock game, where a model guesses a secret word within sixteen attempts, a model trained on summaries failed to close the performance gap with a full-context model, even after reinforcement learning fine-tuning.
This issue stems from the complexity of the learning task. Self-summarising adds a layer of difficulty that standard training data struggles to resolve. Creating effective human simulators for high-quality training environments is also difficult. Consequently, a model’s ability to learn efficient summaries from limited, messy interaction data directly impacts its usefulness for real-world users.
ABBEL: acting through belief bottlenecks
The new framework isolates the summary generation task by treating summaries as belief states. The model updates these beliefs periodically based on new information, drawing on recursive Bayesian estimation methods.
During operation, belief updates alternate with action selection. The agent selects actions based only on the current posterior belief, rather than the entire history.
Belief grading
To improve efficiency, the system extracts and supervises the contents of these belief states. This process, termed belief grading, adds an auxiliary reinforcement learning task. Heuristics define what constitutes a good belief, such as being concise while retaining enough detail to reconstruct the git diff.
Where specific heuristics are hard to define, the authors propose an autoencoding-inspired function. The current language model acts as both encoder and decoder. The system grades each updated belief by how well it allows the model to reconstruct the latest observation.
What do we gain by grading beliefs?
Collaborative coding on CollabBench
The team tested the system in the CollabBench environment, a human-driven assistive coding task. Here, an agent asks clarifying questions before submitting a function to be scored against hidden unit tests.
Using a general reconstruction-based grading function, ABBEL reduced the performance gap relative to full-context models by approximately 50%. Training required half the steps compared to models summarising without grading. After training, ABBEL consumed significantly less memory, measured by peak context token length.
- Full Context Model: Test pass rate of 0.52±0.02; success rate of 0.39±0.02; peak tokens of 14.08±0.55; 100 training steps.
- ABBEL (no grading): Test pass rate of 0.46±0.02; success rate of 0.31±0.02; peak tokens of 4.20±0.37; 100 training steps.
- ABBEL with grading: Test pass rate of 0.48±0.01; success rate of 0.36±0.01; peak tokens of 6.01±0.33; 50 training steps.
Combination Lock
In the Combination Lock test, the team used a belief grader that leveraged domain knowledge. This involved computing useful statistics over the history and verifying they could be reconstructed from the belief state. In this setting, ABBEL approached or exceeded the performance of full-context models while maintaining lower memory usage.
Multi-objective question answering
Finally, the authors tested the system in a multi-objective question answering environment from MEM1. They demonstrated the utility of separating belief states from reasoning. Applying a penalty to peak belief length significantly reduced memory usage with minimal performance degradation.
What it means
For developers building long-horizon agents, ABBEL offers a practical path to reduce context costs without sacrificing capability. By treating summaries as graded belief states rather than simple text, the system learns to retain only the information required to reconstruct the immediate past. This allows models to interact over thousands of steps while keeping memory usage low and training data requirements manageable.
