Foreman: A Self-Hosted, Cost-Aware LLM Gateway for Coding Agents

Foreman is a self-hosted LLM gateway shipped as a single Go binary that sits between a coding agent and any number of model providers, keeping API keys and traffic inside the user's own network. A user-defined policy sorts requests into cheap/mid/frontier tiers, routing high-risk or planning tasks to frontier models while sending routine work to cheaper ones, with the router always picking the cheapest healthy model within an allowed tier.

Using a session ID, Foreman pins a conversation to its current model while the provider's prompt cache stays warm, so cost-driven routing doesn't undermine caching. Provider failures trip a circuit breaker that reroutes traffic, and every request is recorded in a ledger (SQLite by default, Postgres for production) that captures measured cost alongside the reasoning behind each routing decision and the alternatives that were rejected.

CLI tools (top, spend, models, trace) and matching JSON APIs let teams build their own spend dashboards, and cross-protocol tool-call translation lets Anthropic-format clients run against OpenAI-compatible backends like Qwen or GLM. Released under Apache-2.0, the project gives engineering teams a concrete, auditable way to control and reason about LLM spend.