 | According to this. I run several more tests to cover more models and quants. https://www.reddit.com/r/LocalLLaMA/comments/1t53dhp/quality_comparison_between_qwen_36_27b/ Qwen3.6 35B-A3B MLX oQ4. Very very good. (oMLX – local) Qwen 3.6 35B-A3B MLX oQ4’s output is almost perfect. With title, last move label, row and col. But the 2 cursors, one show starting point and the other show end point (red triangles), are a bit confusing at first glance. ZAYA1 8B – Perfect but without a-h, 1-8 row/column mark (Zaya Cloud) ZAYA1 8B is open weight. I used MLX-LM to run it with this PR, but no luck. The 8 bits model kept reasoning in a loop without producing any svg. I don’t think the local inference engine is ready yet. Since the model needs RSA technique to perform. So I posted the result from zaya cloud’s playground – assuming it is FP16 version of it. If somehow local inference engine can produce the same answer, we will have a VERY promising model to run in our tiny computer. The whole process of running 8 bits quant in my computer take less than 12GB of memory. Qwen3.6 27B MLX oQ6. Very good (oMLX – local) no row/no column marks MLX-oQ 6 bit quant of 27B delivered good and correct answer, but no luck pushing to 3.5 bits. Qwen3.6 27B MLX oQ3.5e, Not so good. (oMLX – local) HY3 Preview 295B A21B – Perfect but no line. no row and no column. (Open Router) HY3’s 295B is not gonna cut it on my machine. So the result is from the cloud. Now we’re entering the weird territory – using those thousand derivatives found floating in the hugging face. I’ll be use ones from Jackrong, OrionLLM and DavidAU since all of them published some kind of benchmarks and promise good results. GRM 2.6 Plus Q4K_M – a OrionLLM’s derivative of Qwen3.6 27B – a correct one and looks really good. GRM 2.6 Plus Q3K_M – a OrionLLM’s derivative of Qwen3.6 27B – 3 bits was not gonna cut it. qwen3.6-27b-neo-code-di-imatrix-max@iq4_nl – This 4 bits quant is good. qwen3.6-27b-neo-code-di-imatrix-max@q5k_s – However its 5 bits counterpart was totally wrong. It doesn’t mean that higher bit quant will always perform better than the lower bit ones. Qwopus 35B-A3B-v1 Jackrong’s Q4K_S – the board is wrong and the word game ended came out of nowhere.
|
Key Takeaways
- The 35B-A3B MLX oQ4 model by Qwen is very good, delivering almost perfect results with titles and labels for the moves.
- ZAYA1 at 8 bits does not produce a valid SVG output; it stops reasoning without producing any svg. The FP16 version from Zaya Cloud is used as a reference.
- Qwen3.6 27B MLX oQ6 and Qwen3.6 27B MLX oQ3.5e are both good models, with the latter performing slightly worse.
- The GRM 2.6 Plus Q4K_M model performs well across different bit quantizations (4 and 5 bits), while its 3-bit version fails to produce a valid result.
- Qwopus 35B-A3B-v1’s output is deemed incorrect, with the board being wrong and the word game ending unexpectedly.
Note: This summary is based on the provided information and does not include any new analysis or data beyond what was presented in the original post.
![under 2% quality gap but 10x cost difference: tested 5 models on identical tool calling tasks[D]](https://ai-maestro.online/wp-content/uploads/2026/05/under-2-quality-gap-but-10x-cost-difference-tested-5-models-768x768.jpg)
