triton

How Netflix Built Its Own In-House LLM Serving Stack

Netflix details its in-house LLM serving architecture: vLLM engine choice, Triton packaging, OpenAI-compatible API, and production rollout issues.