Vosti: Specifying, Implementing, and Verifying Deterministic LLM Inference

Vosti is an inference engine designed to ensure determinism in LLM inference systems. Under a fixed model and deployment configuration, it guarantees that requests with the same prompt and initial state yield bitwise-identical outputs. Tests reveal that existing production modes may produce different outputs under certain execution variations, highlighting the need for a formal specification. Vosti addresses this issue with a verified approach, achieving performance comparable to existing systems while ensuring stronger determinism guarantees.