» Tag
testing
87 postsWhy your eval set is too small to catch real regressions
Small eval sets miss most regressions. Learn how statistical power, Wilson intervals, and binary criteria fix eval-driven development.
7 Regression Tests Every AI Agent Should Pass Before Deploy
Discover essential regression tests for AI agents. Ensure a successful deployment with these key evaluations.
Setting Up E2E Tests with Playwright and Auth0: 5 Time-Consuming Steps
Jo4 shares challenges and solutions encountered while setting up E2E tests with Playwright.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comCoding with AI: The Importance of Architectural Tests
Key insights on the necessity and effective use of architectural tests when coding with AI.
Invalid Tests: Why 'n/a' Is Not a Verdict
The validity of tests depends on hardware presence. The 'n/a' status offers important lessons for engineers.
Browser Test Reliability: The Shift to Product Decision-Making
Browser test reliability has evolved into a product decision-making issue due to modern application complexities. A holistic approach is now essential.
The Modern Browser Testing Stack: AI, CI, Human Review, and Maintenance Costs
Modern browser testing involves AI and CI complexities. The article discusses flaky test costs and the importance of accurately reproducing test environments.
Building a Transpiler to Migrate Over 4,000 Tests to Playwright
The migration of over 4,000 tests to Playwright highlights the need for a transpiler. This transition is vital for enhancing QA efficiency.
Why You Need a Smoke Test Before Switching LLM Providers
Sending one prompt isn't enough before swapping LLM providers. A 10-minute smoke test covering timeouts, streaming, rate limits, and tool-call retry safety.
Testing Distributed Systems with AI Coding Agents
Two SKILL.md based agent skills generate claim-driven test plans and verdict reports for distributed systems, working with Claude Code, Codex, and more.