» Tag
testing
96 postsContract-Testing API Emails in GitHub Actions Pipelines
A practical guide to why email tests break in CI and how to contract-test API-triggered emails in GitHub Actions using run-scoped inboxes.
Why Your LLM-as-Judge Gate Flips Between Runs
Same code, same input, different LLM judge score: why CI gates flap between runs, and how temperature, model pinning, k-sampling and quantization fix it.
Your LLM Judge Needs a Regression Test Suite
rubric-bench v0.1 is an open-source tool that regression-tests LLM judges against golden sets, catching grading drift in CI before it reaches production.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comParlel Emulates 250+ Services Locally With Real Wire Protocols
Parlel runs 250+ services like Postgres, Redis, Stripe and S3 locally using real wire protocols, offering a free, fast alternative for testing and AI coding agents.
A Run ID Pattern for Stable React Auth Email Tests
Flaky React auth email tests usually stem from missing correlation, not timing. A typed run id pattern ties UI state, API, and inbox checks together for reliable assertions.
Spin Up Ephemeral Test Inboxes for Email Integration Tests
Discover the advantages of using ephemeral test inboxes for email integration testing.
How to Stop Duplicate React Mention Emails
A look at why React mention emails get sent twice and how a shared eventId contract between UI, API and worker keeps notifications idempotent.
Inbox Contracts: A Testing Pattern for Scheduled Email Jobs
A practical pattern for scheduled jobs: isolate each run with its own mailbox, run id, and evidence-based assertions to catch silent email bugs early.
Why LLM-as-judge scores flip between identical runs
An LLM judge gate flipped between 0.79 and 0.82 on identical inputs. Sampling temperature, model drift and vague rubrics cause the jitter; k-sampling and a noise band fix it.
Data Races and the Limits of ThreadSanitizer
Learn about data races and the limitations of ThreadSanitizer in C and Go applications. Key insights for engineers.