An Error Notebook for AI Agents: 266 Rules, 66 Catches

A developer building on the opencode agent noticed his AI coding assistant kept repeating the same mistakes, exposing a gap in the usual CLAUDE.md/AGENTS.md approach: these files store static project preferences but never capture dynamic lessons from past errors. In one striking case, the agent claimed to have modified 172 files by conflating 'generated a script' with 'actually changed files' — zero files had been touched.

His fix was to replace negative rules ('don't do X') with action-based ones ('when X triggers, execute Y'), since agents can recognize a condition far more reliably than reconstruct an entire failure pattern from a prohibition. Over two months he accumulated 266 such rules and split them into three tiers: 14 core rules auto-injected every session, 88 task-specific rules loaded via keyword matching, and 164 archived rules searchable on demand. The whole system runs as a feedback loop — correction trigger, rule extraction, boot-time injection, real-time interception, and effect tracking that retires underperforming rules.

The results are notable: one error pattern, 'skipping the manual before starting work,' fired 66 times and was caught 94% of the time, suggesting it's a deeply ingrained behavioral tendency occurring roughly once every two to three sessions. The author is transparent about limitations too — the data is self-reported from a single user, model improvement over time can't be ruled out as a confounder, and most of the 164 archived rules barely fire at all. Still, the project offers a concrete example of turning an AI agent's context file from a static preference list into a measurable, self-improving error-prevention system.