Agentic Context Engineering (ACE) sends its full playbook to the model at every step, consuming hundreds of thousands of tokens per task. IBM’s new system, ALTK-Evolve, achieves equal or better accuracy while using just a fraction of that context. It does so by selecting only the guidelines a specific model can actually use.
The memory problem
When an LLM agent tackles a realistic multi-step task, failure usually stems from a lack of knowledge. The agent might mispaginate an API, resolve the wrong person, or return a value when none was asked for. The model knows the APIs; what it has not internalised is how to use them reliably. That is learnable from the agent’s own history.
Two recent systems do exactly this on the same kind of agent: ACE and ALTK-Evolve. Both are a form of agentic memory, turning an agent’s past trajectories into reusable lessons and feeding them back at inference time without weight updates or human labels. They agree on the hard part. Where they part ways is delivery.
A note on words, because the two systems name things differently: we will call the raw thing an agent learns a lesson. ACE organises its lessons into one comprehensive, evolving playbook; we consolidate ours into individually retrievable guidelines. Same lessons, two containers.
What we agree on
Both systems refuse to compress.
ACE names the failure modes precisely: brevity bias, where optimisation collapses towards short, generic instructions, and context collapse, where a model asked to rewrite its whole context each step summarises the detail away. Its answer is to keep a rich, itemised playbook, with a helpful/harmful counter on every bullet, and let the model distill relevance at read time.
We reach the same conclusion from the other direction. Every distinct guideline keeps a support count, representing how many independent episodes produced it, and we never summarise the store down to a handful of rules. A lesson five different tasks discovered is a different object from one that appeared once, and both are worth keeping.
So on the core question, should you compress an agent’s hard-won lessons into a tidy summary?, ACE and ALTK-Evolve give the same answer: no. Count them, don’t collapse them. ACE’s per-bullet counters and our support counts are two spellings of the same idea.
Where we differ
Two places: how the memory is built, and how it’s delivered, and it’s the delivery difference that shows up in the token bill.
Consolidation, how the store is built. ACE grows one playbook through a Generator, Reflector, Curator loop, applying incremental delta updates and de-duplicating by embedding. We cluster near-duplicate lessons and merge within a cluster, support-conserving, so the store shrinks without losing the record of how much experience backs each guideline. We also extract typed guidelines, strategy, recovery, and optimisation, with causal attribution and provenance back to the source trajectory, at subtask granularity, so a lesson learned on one app can transfer to another.
Delivery, what reaches the model at inference. This is the one that drives the numbers. ACE injects the comprehensive playbook on every step, the same way regardless of model or task. We treat delivery as a dial, not a constant: a small fixed core of high-support guidelines, extended per task with a handful selected for the task at hand via cosine or LLM-guided, priority-weighted selection, or, when a model has the headroom to use it, the full consolidated set. The same lessons are available to both agents; the difference is that ACE always sends all of them, and we send however many a given model can actually use.
Why it matters
On AppWorld, with the same base ReAct agent, running both systems in-house:
| Model | System | TGC / SGC | Tokens/task |
|---|---|---|---|
| DeepSeek-V3.2 | ACE | 80.4 / 73.2 | 634K |
| DeepSeek-V3.2 | ALTK-Evolve | 89.3 / 80.4 | 263K |
| gpt-oss-120b | ACE | 54.8 / 35.7 | 777K |
| gpt-oss-120b | ALTK-Evolve | 56.0 / 37.5 | 116K |
On the strong model we’re better on both metrics at roughly 40% of ACE’s inference cost. On the weak model we edge ACE 56.0 to 54.8, close enough that we call it a tie on accuracy, as a repeat run of ours landed at 54.8, matching ACE almost exactly, which is within this benchmark’s run-to-run noise, at about one-seventh the cost.
A fair word on cost: ACE’s own efficiency story is about building its context cheaply. Ours is on a different axis, serving it. Retrieving a few guidelines per task instead of injecting the whole playbook on every step is where the tokens go, and it’s the direct consequence of the delivery difference above.
Where does the accuracy come from? The by-difficulty breakdown tells two different stories:
Figure 1 shows Post-memory Task Goal Completion by difficulty, ours vs. ACE. On DeepSeek-V3.2, we win Easy, Hard, and Overall; ACE only edges Medium. On gpt-oss-120b, ACE leads easy and medium, but per-task selection wins the hard tasks and the aggregate. Each system improves from its own no-memory baseline.
The two models tell different stories. On gpt-oss-120b, ACE’s full playbook has the edge on Easy and Medium, as there is enough of the task solved by generic instruction-following that a comprehensive prompt helps more than it distracts. But on Hard tasks, where the model has to pick the right lesson rather than wade through all of them, curated retrieval pulls ahead, and that’s the tier that decides the aggregate. On DeepSeek-V3.2 the story flips: the stronger model absorbs ACE’s full playbook well enough to edge us on Medium, but we lead Easy, Hard, and Overall, with more capacity to spare, more lessons keep helping instead of crowding each other out.
We give each model its best configuration, the full consolidated set for the strong model, selective retrieval for the weaker one, because a large context overwhelms a weaker model rather than helping it.
Same lessons, different delivery
Both systems refuse to compress an agent’s hard-won experience into a tidy summary, that part we agree on. The difference is whether delivery is fixed or calibrated: ACE sends the whole playbook every step no matter what; we send however much of the guideline set a given model can actually use. That calibration is what bought the numbers above, same-or-better accuracy at a fraction of ACE’s inference cost, and on the weaker model, it was the difference between guidance that helped and guidance that got in the way.
What it means
For people building agents, the distinction is practical. If you run a large model, the full context of ACE may work, but it wastes tokens. If you run a smaller model, the full context becomes noise. ALTK-Evolve adapts to the model’s capacity, keeping the agent smarter without forcing it to read irrelevant history.
Linked artifacts / references
- Earlier post: ALTK-Evolve introduction
- ACE (Agentic Context Engineering)
- AppWorld benchmark
- ALTK-Evolve
- Full technical report
Method notes
AppWorld test_normal, 168 tasks. A ReAct code agent, each step writes Python, the environment returns the output. TGC is Task Goal Completion; SGC is Scenario Goal Completion, which requires every variant of a scenario to pass. Memory is mined from train/dev only; results are single runs, pass1, as is standard on this benchmark.
The ACE numbers are our own runs of the ACE agent, evaluated in-house on the same AppWorld splits and the same base models as ALTK-Evolve, DeepSeek-V3.2 and gpt-oss-120b. The ACE paper reports on a different base model, DeepSeek-V3.1, so running it ourselves keeps the comparison controlled for model and harness. Both systems are the same ReAct agent and differ only in the prompt template, which is why the two no-memory baselines differ, 72.0 vs 79.8 TGC; we do not rest the comparison on that baseline gap, only on the claims a prompt tweak cannot touch, same-or-better accuracy at a fraction of the tokens.
Reference tables
DeepSeek-V3.2 test_normal, 168 tasks:
| System | Guidelines | TGC | SGC | Tokens/task |
|---|---|---|---|---|
| ReAct, no memory | 0 | 79.8 | 64.3 | 148K |
| ReAct + ACE | 106 | 80.4 | 73.2 | 634K |
| ReAct + ALTK-Evolve | 191 | 89.3 | 80.4 | 263K |
gpt-oss-120b test_normal:
| System | Guidelines | TGC | SGC | Tokens/task |
|---|---|---|---|---|
| ReAct, no memory | 0 | 39.9 | 21.4 | 110K |
| ReAct + ACE | fullSource Read original →Related articles  |