Intel Arc B70 32GB Runs Qwen3.6-35B-A3B at 130 t/s via Vulkan
Benchmark: Intel Arc Pro B70 32GB runs Qwen3.6-35B-A3B via llama.cpp Vulkan at ~130 t/s (4-bit, 262k ctx) and ~69 t/s hybrid 8-bit offload.
A community benchmark on Intel's Arc Pro B70 (32GB, Battlemage) shows llama.cpp's Vulkan backend running the Qwen3.6-35B-A3B MoE model at practical speeds for local inference. With the Q4_K_XL quant (21.3GB) fully resident in VRAM, the card sustains ~130 tokens/sec at the model's full 262k context window using an f16 KV cache — which outperformed a quantized q8_0 cache in this setup.
For the larger Q8_K_XL quant (36.4GB, exceeding the 32GB VRAM), a hybrid setup offloading MoE expert layers to system RAM via --n-cpu-moe found an optimum at 12 offloaded layers (69.4 t/s); pushing further offload triggered a silent VRAM spill that collapsed throughput to ~27 t/s.
The GGUF's embedded MTP (multi-token prediction) draft head, enabled via --spec-type draft-mtp, delivered a 1.66x speedup (32.5→54.1 t/s) with 83% acceptance — beating classic separate-model speculative decoding, which underperformed baseline on this MoE architecture. Tests also confirm Vulkan substantially outperforms SYCL for MoE workloads on Battlemage (~130 vs ~70 t/s), attributed to SYCL's expert-routing kernel disabling graph capture and forcing per-layer host synchronization.
For engineers evaluating Arc Pro cards for local LLM serving, the results highlight three practical takeaways: keep GPU drivers current (older builds crashed under MTP plus long context), treat quantized KV cache as a memory-saving option rather than a speed one, and expect throughput to roughly halve at 30k+ context depth regardless of configuration.