» Tag
ci-cd
41 postsTesting LLMs Like Software: A Promptfoo Deep Dive for QA
A production-level guide to testing LLM systems with Promptfoo: versioned evals, deterministic checks, judge-model bias, and provider benchmarking.
Probelock: A Lockfile for LLM Tool-Calling Capabilities
Probelock measures an LLM's tool-calling capabilities with deterministic tests and halts CI when a model, quantization, or runtime swap causes a regression.
Kakehashi: run macOS ARM64 binaries on Linux aarch64, no JIT
Kakehashi is an experimental userspace layer that runs macOS ARM64 binaries natively on Linux aarch64 without JIT, aimed at cutting CI costs.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comBuilding an Arch Linux Aarch64 Port for Holo Core
Collabora and Valve detail Holo Core, an unofficial aarch64 port of Arch Linux for Steam Frame, and the CI tooling needed to replay its build history.
17 PRs a Day, One QA: Automating E2E Failure Triage
How pdf.net automated e2e failure triage with a Claude-powered GitHub Action, letting one QA engineer keep pace with 17+ daily merges.
Unfinished Work in Package Security: Gaps Between Controls
Nx, Ledger, and tj-actions incidents expose gaps between independent package security controls, with lessons for engineering teams.
The physics of Docker build caching: three caches, one wave
A measured breakdown of Docker's three build caches — layer chain, mount cache, image store — and when each caching strategy actually helps.
eBPF and TCX bring line-rate network visibility to hostile CI jobs
Blacksmith explains how eBPF programs attached via TCX and a host-side DNS proxy deliver line-rate network observability for untrusted CI VMs.
30 Days as the Only Human in an AI Dev Team: 212 PRs, Broken CI
A 30-day AI-agent coding experiment shows how self-reported 'green' CI and unreviewed PRs let critical failures hide in plain sight.
Loop Engineering: Fixing a Guardrail That Fired Wrong
How to fix a misfiring guardrail check in an agent loop without deleting it—why weakening checks silently disables real catches.