PrismML has released Ternary Bonsai 2 27B, a model weighing 5.93 GB that retains 98.2% of the performance of the 53.80 GB FP16 Qwen3.8 27B baseline. The release arrives two months after the original Bonsai 27B, which kept 95% of its parent model’s scores. The new version accepts text and images, supports a 262K-token context, and runs on a single 24 GB GPU or a 16 GB laptop using PrismML’s custom llama.cpp fork or MLX runtime.
What is Ternary Bonsai 2 27B?
The architecture mirrors Qwen3.8 27B with 27.36B parameters split into a 24.35B language backbone, 2.54B for embeddings and the language model head, and 0.47B for the vision tower. The backbone uses hybrid attention, combining roughly 75% linear-attention layers with 25% full-attention layers.
Ternary weights apply to the embeddings, attention projections, MLP projections, and the language model head. Only 26.2M parameters, or 0.0976%, remain in higher precision. These high-precision weights cover the recurrent state path and normalization layers. In GGUF format, the vision tower ships as a separate 0.63 GB file and loads only when processing images.
How Does the Ternary Format Work?
Each weight takes one of three values: -1, 0, or +1. Every group of 128 weights shares a single FP16 scale. A ternary value carries approximately 1.585 bits, calculated as log2(3). Adding 16 scale bits per 128 weights results in 1.71 bits per weight. Including the high-precision tensors brings the total to 1.72 bits per weight.
Real kernels require a packed layout. The accompanying whitepaper describes two GGUF packings. PTQ1_0 packs trits densely at 1.76 bits per weight for a total size of 5.93 GB. PQ2_0 stores each trit in a 2-bit slot, increasing the size to 7.25 GB but making unpacking cheaper.
Weights are stored in a rotated basis. PrismML applies a blockwise Hadamard rotation with a block size of 1,024 before ternary assignment. The runtime applies the matching transform to activations before each multiplication. The whitepaper cites SpinQuant for this technique. PrismML does not publish the method for assigning ternary values.
How Does It Score Against Qwen3.8 27B?
PrismML evaluated all models in thinking mode using EvalScope and vLLM on H100 GPUs. The comparison includes Qwen3.6 27B as a reference point.
| Capability | Qwen3.6 27B | Qwen3.8 27B | Ternary Bonsai 2 27B | Retention |
|---|---|---|---|---|
| Knowledge and reasoning | 84.71 | 86.66 | 83.95 | 96.9% |
| Math | 94.64 | 97.06 | 96.57 | 99.5% |
| Coding | 82.57 | 82.17 | 81.58 | 99.3% |
| Agentic and tool calling | 80.05 | 79.74 | 77.57 | 97.3% |
| Instruction following | 74.53 | 81.25 | 82.66 | 101.7% |
| Vision | 79.82 | 81.64 | 78.59 | 96.3% |
| Overall (20) | 83.6 | 85.4 | 83.9 | 98.2% |
The comparison with conventional quantization shows a sharper result. An IQ2_XXS build of Qwen3.8 27B averages 75.2 at 7.3 GB. On AIME26 it scores 78.6, while Bonsai 2 scores 95.83. On LiveCodeBench v6 the gap is 70.05 versus 90.07.
Where Does It Still Lose Quality?
The 98.2% figure is an average, and the losses are uneven. Vision retains 96.3% and knowledge and reasoning retains 96.9%.
Long-horizon agent work drops further. Bonsai 2 scores 52.8 on Terminal-Bench 2.1, against 69.7 for Qwen3.8 27B. On SWE-bench Verified it scores 60.8 against 80.6. That is about 75% retention, and both sit outside the 20-benchmark average.
Reasoning effort matters too. At medium effort the model averages 79.3, against 82.6 for the FP16 baseline. Low effort is not supported. All results are PrismML’s own and have not been independently reproduced.
How Fast Is It on Real Hardware?
Figures are batch size 1 decode on PrismML’s custom kernels, measured September 16, 2026. An RTX 5090 reaches 142.5 tokens per second at 0.582 mWh per token. An RTX 4090 reaches 96.7 with PTQ1_0, and a 72 W L4 reaches 32.1. On Apple laptops, an M5 Max reaches 46.8 and an M5 Pro reaches 27.7.
Neither packing wins everywhere. PTQ1_0 is faster on Ada-generation cards and the L4. PQ2_0 is faster on Blackwell, Hopper, Ampere and Apple silicon, and at prompt processing everywhere.
PrismML research team also claims 40% better energy efficiency than a full-precision 8B model.
How Do You Run It?
The GGUF files need PrismML’s llama.cpp fork. Stock llama.cpp rejects the PTQ1_0 and PQ2_0 types. The Bonsai-demo repo is the supported path. Run ./setup.sh, then ./scripts/start_llama_server.sh for chat, vision and tools at localhost:8080.
Mac users can take the MLX pack, which needs its bundled loader. A WebGPU demo runs the model inside a browser.
What it means
Developers can run a large language model on a laptop with 16 GB of RAM. The 5.93 GB footprint allows the model to fit in memory alongside other tools. Users do not need a cluster to test agentic workflows or run coding assistants locally. The trade-off is that complex, multi-step tasks lose about a quarter of their original capability.
