« All posts

Local Model Showdown Round 9: Qwen 3.6, Nemotron, Qwythos Coding Test

Five LLMs face off on a real coding task via llama.cpp on an RTX 5090 in Round 9 of the Local Model Showdown series.

In Round 9 of the Local Model Showdown series, five models were benchmarked on a single RTX 5090 via llama.cpp against a real coding task: building a full tag manager feature for an admin panel. A planned three-way dense-vs-MoE test expanded to five contestants after two models hit hard failures on their very first request, both traced to llama.cpp's automatic tool-call parser colliding with each model's embedded jinja chat template — requiring live extraction and hand-patching of the templates from GGUF metadata mid-bakeoff.

Three of the five models shipped a working, mergeable PR. Qwen 3.6 35B-A3B, which had spiraled for 77 messages without committing in an earlier round, returned with the cleanest run of the round, suggesting its prior failure was situational rather than architectural. The dense 27B challenger also completed the task but needed roughly twice the interventions. Qwythos-9B and Nemotron-3-Nano failed even after their template bugs were fixed, each for a distinct reason unrelated to the original parser issue.

For engineers, the key takeaway is twofold: the dense-vs-MoE hypothesis got murkier rather than clearer — the round's best run was an MoE model, and failures split evenly across architectures — and infrastructure-level template/tool-call handling in llama.cpp can silently derail a model's behavior before a task even begins.