For the maker and the artist, the shift from a passive observation deck to an active financial simulation offers a new playground for narrative generation. The original Thousand Token Wood was merely a sandbox where five woodland creatures, driven by a single 0.5B model, reacted to external shocks. You watched bubbles form and crash without direct influence.
The second iteration transforms this into a playable game of shadow banking. You assume the role of the Patron, a financier who lends at interest, plants misinformation, shorts the market, and brokers alliances while evading a magistrate’s scrutiny. The creatures retain memory of your actions and retaliate accordingly. The core engineering shift lies beneath the surface: every creature now operates using a distinct small model from a different laboratory.
Heterogeneity is the product, not a constraint
The standard approach to running a council of agents involves one model with varied prompts. This version deploys four distinct architectures: gpt-oss-20b from OpenAI, MiniCPM3-4B from OpenBMB, Nemotron-Mini-4B from NVIDIA, and a custom fine-tuned Qwen 0.5B. The goal is not novelty, but genuine divergence. A market becomes compelling only when participants possess truly different perspectives. These four models, trained on disparate data and subjected to different post-training methods, represent the maximum diversity currently achievable with small parameters. The owl manages assets differently than the fox speculates, turning the council into a live debate rather than a scripted event.
Deploying these four distinct models on a single platform revealed a critical insight: the friction resides almost entirely at the serving layer, not within the models themselves.
- Current vLLM version 0.22.1 JIT-compiles kernels upon load and requires the CUDA toolkit (
nvcc
) to be present. Since lean base images do not include it, all four models initially failed with a “could not find nvcc” error. This was a universal issue with that vLLM version, not a specific model quirk. Switching to a CUDA development image resolved the problem for everything.
- gpt-oss-20b operates in its native MXFP4 quantization and fits comfortably on a 24GB L4 GPU with room to spare, eliminating the need for high-end hardware. Additionally, it uses a channel format that wraps the final answer in an analysis preamble, requiring the consumer to extract the specific output.
- MiniCPM3 required enabling
trust_remote_code
, whereas Nemotron loaded without issues. These were isolated configuration footguns, each solvable with a single line of code.
The element that made running four heterogeneous models manageable was the same primitive used in version one: a tolerant JSON parse-and-repair layer. Every model’s output flows through this filter. Because different tokenizers and formatting habits produce various malformations, the parser discards unsalvageable data, ensuring the simulation never crashes. Building this layer once means adding a new model is simply a configuration entry, not a major refactor.
Information asymmetry needs a firewall
The dramatic core of this version revolves around insider tips. You can whisper a tip to a creature that is either true (a genuine forecast of the next market mania) or false (bait). Profiting from a true tip raises your “heat”; crossing a threshold triggers a magistrate investigation leading to fines, frozen assets, or exile.
For this to function as a real game, the truth of the tip must remain hidden from the creatures. They see the text of the rumor but must never see the flag indicating its validity. This is a security property, not a user interface nicety. Small-model agents make this critical: whatever is in their prompt is whatever they can repeat. Therefore, the hidden flag lives entirely outside the prompt (on the player’s ledger), is stripped from the public event record at construction, and the narrator only summarises public events. A single test scans every creature’s full prompt, turn by turn, for banned tokens. This test is paramount. When you provide an agent with secret information, assume it will leak unless a test proves it cannot.
Memory is cheap drama if you bound it
Characters maintain persistent relationships: a signed sentiment toward the Patron and toward one another, influenced by events such as loan repayments, crop shorting, or forming alliances. A creature that turns hostile refuses loans and demands worse terms; allied creatures stop undercutting each other and behave like a cartel.
The trap is prompt inflation. Raw history grows indefinitely, causing a small model to drown in context. The solution is to never put full history in the prompt. Instead, the model sees a one-line bucketed summary, such as “you feel warmly toward Oona, wary of the Patron,” capped to the few strongest feelings derived from integer sentiment. Notes are kept for tracing but remain bounded and invisible to the model. The resulting behavioural bias is part emergent (the summary nudges the model) and part mechanical (a strongly hostile creature deterministically refuses), making it observable and testable rather than a hope.
What actually happened
A representative council run, with the full v2 mechanics active:
| Lever | Result |
|---|---|
| Models in the council | 4 labs, all under the 32B cap, served on Modal |
| Fine-tuned 0.5B reliability | 0% self-buys, 100% valid offers (beats its 3B teacher) |
| Truth firewall | 0 leaks of a tip’s hidden flag across every prompt scanned |
| Insider tip edge | a true-tip pre-position settles a positive P&L; a false tip does not |
| Heat to investigation | two clean suspicious wins cross the magistrate’s line |
| Ruin | a margin call and a loan default banish a creature, who returns a chapter later |
This represents a single seeded run exercising the Patron, the information war, relationships, and leverage end to end.
Takeaways for building with small models
- A small model is a reliable format generator and an unreliable reasoner; you close the gap with structure, prompting, and a small fine-tune, not with scale.
- A heterogeneous council is more interesting than a homogeneous one and costs you only config once the serving layer is solid.
- Secret information given to an agent is a firewall problem, and the firewall belongs in the data flow, proven by a test, not in a prompt instruction.
- Persistent memory is the cheapest way to make agents feel alive, as long as the prompt only ever sees a bounded summary.
