Transformers vLLM Backend Now Matches Native Inference Speed

Hugging Face has upgraded the transformers modeling backend for vLLM so it now matches or beats hand-written, model-specific vLLM implementations in throughput. Benchmarks across Qwen3-4B (single GPU), Qwen3-32B (tensor-parallel), and Qwen3-235B-A22B-FP8 MoE (data+expert-parallel on 8×H100) show the transformers backend meeting or exceeding native vLLM speed in every case.

The gain comes from a new optimization pass: the backend uses torch.fx to statically analyze a model's compute graph, detects known fusable patterns (such as MergedColumnParallelLinear, QKVParallelLinear, and MoE expert-parallel kernels), and rewrites the source via ast at runtime. The resulting graph remains fully compatible with torch.compile and CUDA Graphs, so nothing about deployment setup changes.

For model authors, this means any compliant Hugging Face architecture can hit native vLLM inference speed with a single flag (--model-impl transformers), across tensor, pipeline, and expert parallelism, without writing custom inference code. The same model code can also be reused for training, evals, and RL rollouts. Current limitations: linear-attention architectures aren't supported yet, and fully custom Hub-only model code likely won't work since it wasn't written to the required conventions.