AI systems discard user instructions when they summarise conversation history to save space. On average, only 17 percent of these constraints survive the process.
Context windows fill up quickly when users keep a single chat open for long periods. To manage this, labs use compaction, which summarises previous messages to free up room for new input. The trade-off is that the summary inevitably strips away details.
Researchers at Penn State have now examined exactly which details disappear and how severe the impact is.
User restrictions are the first thing to go
The primary casualty is what the study calls “session constraints”. These are rules governing the AI’s behaviour for the duration of a specific chat, such as “Confirm with me before making any changes” or “Never use my name in your responses”. They are not permanent system settings, nor are they part of the core task itself.
This makes them fragile. Compression tools prioritise preserving the main goal, current state, and next steps. User-imposed side conditions get dropped.
A person who says “Don’t send any emails without my approval” must expect that the agent will ignore this after compaction. This is both a quality and a security problem. Agents can end up making unauthorised tool calls, disclosing withheld information, or skipping verification steps the user explicitly required.
To measure the loss, the team created an evaluation suite called COMPINT. Only 17 percent of injected session constraints survive compression on average. Most tested compactor setups perform worse than running without compression at all, though GPT-5.4-mini beats the baseline in some scenarios.
When the agent receives the full, uncompressed context with the user constraint intact, rule compliance sits between 59 and 71 percent, according to the study. After compression, compliance drops sharply for most tested compactors and often lands only slightly above the level seen when no constraint was given at all. More targeted compression prompts help but do not close the gap. Even a prompt built specifically to preserve user constraints keeps retention below 40 percent, the researchers say.
A small language model offers a plug-and-play fix
What actually helps is a small add-on module that runs alongside the main compression system, the researchers say. It is built on Qwen3.5-9B, a compact language model, and reads every user input to detect session constraints and collect them in a separate list. When the context gets summarised later, the module appends that list to the summary so the user’s rules stay intact.
The extractor hits over 90 percent retention across all three tested scenarios, according to the study. That breaks down to 95.6 percent for agent trajectories, 95.1 percent for long-term research tasks, and 90.3 percent for multi-turn chats. It needs no training and no changes to the compression system itself.
The COMPINT evaluation suite and the extractor are available on GitHub.
