» Tag
llm
478 postsRelay: A Self-Hosted LLM Gateway With Eval-Gated Smart Routing
Relay is a self-hosted, single-binary LLM gateway with zero telemetry and an eval harness that gates smart routing by default.
LLMrPro: Open-Source LLM Balancer Pools Local Machines With Cloud Fallback
LLMrPro is an open-source, self-hosted LLM balancer that unifies your own machines and cloud providers behind one OpenAI-compatible API.
AI's Cost Paradox: Smarter Models, Yet Runaway Token Bills
AI models keep improving, yet 'token amplification' in agentic workloads is driving usage costs sharply upward, forcing firms to curb AI spending.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comHow Netflix Built Its Own In-House LLM Serving Stack
Netflix details its in-house LLM serving architecture: vLLM engine choice, Triton packaging, OpenAI-compatible API, and production rollout issues.
AI agents are shifting from loops to graphs
An analysis of the shift from loop-based AI agents to graph-structured architectures, with practical guidance for engineers.
Harness Engineering: Curating Context to Scale AI Coding Agents
Harness engineering keeps AI models fixed and optimizes context and tools instead, encoding nonfunctional requirements for reliable agent output.
WANDR Benchmark Tests AI Agents on Wide-and-Deep Research Tasks
WANDR benchmark evaluates AI research agents on wide-and-deep data collection tasks using reference-free, evidence-verified grading across 500 tasks.
How to Become a Forward Deployed Engineer in 2026
A concrete 2026 roadmap to becoming a Forward Deployed Engineer: key skills, portfolio artifacts, interview stages, and a realistic prep timeline.
How a shared JSONL file inflated our eval score to 0.94
A retrieval-based few-shot selector shared its index with the eval set, inflating accuracy from 0.79 to a fake 0.94. Here's the contamination check that catches it.
DiffusionGemma: Google's Diffusion-Based LLM Developer Guide
Google's DiffusionGemma developer guide: diffusion-based LLM with bidirectional attention, 4x faster generation, vLLM support, and Sudoku fine-tuning results.