» Tag
llm
536 postsJean2: A Coding Agent That Evolves Across Sessions via File-Based Memory
Jean2 is a coding agent that writes what it learns into markdown files via opt-in memory, skill and session-search layers, improving through accumulated knowledge rather than model retraining.
Why Your LLM-as-Judge Gate Flips Between Runs
Same code, same input, different LLM judge score: why CI gates flap between runs, and how temperature, model pinning, k-sampling and quantization fix it.
TaxCalcBench: Open Source Test for AI Tax Filing
TaxCalcBench is an open source benchmark testing whether LLMs can accurately file real tax returns; even the best model only reaches 54% full accuracy.
CommitBrief — AI code reviews, right in your terminal
A provider-agnostic, local-first CLI that reviews your staged changes, a historic range, or a whole GitHub pull request. Zero telemetry, no server. Free and open source.
commitbrief.comRegex Beats LLM Classifiers by 45 Points in Agent Context Routing
In a new test, plain regex rules beat TF-IDF classifiers by 45 points at routing agent context, showing an LLM call is often unnecessary.
When Fine-Tuning Stalls: The Architecture That Took Recall to 92%
A healthcare claims AI stuck at 45% recall on CPT/ICD coding reached 92% by adding a deterministic NCCI validation layer instead of more fine-tuning.
What Breaks MCP Servers in Production
A look at real production failures in MCP servers—model-unreadable errors, token rotation logouts, schema key rejections, size limits—and the standard built to fix them.
Deprecated Accessor Trap in AI SDK v7 Silently Erases Agent Memory
In AI SDK v7, result.response.messages now returns only the final step, silently dropping tool calls from history and crippling multi-turn agent behavior in smaller models.
Your LLM Judge Needs a Regression Test Suite
rubric-bench v0.1 is an open-source tool that regression-tests LLM judges against golden sets, catching grading drift in CI before it reaches production.
Local AI Pipelines: Why Some Workloads Should Never Leave
Privacy, cost, and latency push certain AI tasks off the cloud and onto local hardware. A practical local AI pipeline using Ollama and faster-whisper.
Metis: Arm's Open-Source Agentic Security Review Tool
Metis, built by Arm's Product Security Team, is an open-source framework that uses LLM reasoning to find security vulnerabilities across large codebases.