I was frustrated that every coding agent (OpenCode, Cursor, Claude Code) assumes you’re running GPT-5.4 or Claude Opus. If you try them with a local model like Gemma or Qwen they fall apart. I find that often tool calls fail, context overflows, multi-step tasks collapse.
So I built SmallCode. It’s designed from the ground up for small local models.
The result: 87/100 benchmark tasks pass with a Gemma 4 model that only activates 4B parameters per token. OpenCode scores ~75% with 14B models. The harness does the heavy lifting, not the model size.
How it works (the tricks that make small models reliable):
- Compound tools: Instead of making the model chain 4 tool calls (find file → read file → edit file → verify), SmallCode gives it one tool that does all 4. Small models lose coherence after 3+ sequential calls. This cuts failures in half.
- Improvement loop: Every time the model writes code, SmallCode instantly compiles/lints it. If it fails, it feeds the errors back automatically. The model doesn’t need to be smart enough to get it right first try, it just needs to fix errors when shown them.
- Decompose on failure: If the model fails the same thing twice, SmallCode stops retrying and instead breaks the problem into smaller pieces. "Fix this 200-line file" becomes "fix line 45 only."
- Escalation: If even decompose fails and you have a Claude/OpenAI key configured, it auto-escalates to the bigger model for just that one task. You stay local 95% of the time, cloud 5%.
- Token budgeting: Small models have 32k-256k context. SmallCode never dumps a whole file in. It summarizes, truncates, and manages every token so the model never sees "…" truncation in the middle of important code.
- Code graph: Instead of grep-searching your codebase, SmallCode indexes your code into a symbol graph (functions, classes, who-calls-what). When you ask "how does auth work," it walks the graph and returns just the relevant connected code, not 15 random file snippets.
What it looks like:
Full-screen terminal UI (like OpenCode/vim), scrollable chat, command palette with /, plugin system, persistent memory across sessions.
What it doesn’t do:
- No LSP integration (yet)
- No multi-session (yet)
- No desktop app
- Doesn’t compete with Claude Code for frontier model users
Install:
npm install -g smallcode cd your-project smallcode
Point it at LM Studio, Ollama, or any OpenAI-compatible endpoint.
MIT licensed, everything’s on GitHub: https://github.com/Doorman11991/smallcode
Happy to answer questions about the architecture or benchmark methodology.
