My AI Reviewer's Real Problem Was Sequencing, Not Rules
A writer's AI-assisted editorial reviewer kept failing in new ways until distinct reasoning tasks were staged as separate passes instead of expanding the rubric.
The author is building an AI-assisted editorial pipeline that turns Notion cards into markdown drafts, runs them through review, and syncs them to dev.to. An earlier fix had reordered the reviewer to analyze before scoring, ending premature QA-style feedback. But three new failure modes surfaced afterward: critiques that agreed with themselves too easily, drafts that kept growing instead of tightening, and sections that drifted into becoming their own mini-articles.
Each time, the instinctive fix was to expand the rubric or lengthen the prompt — and each time that was the wrong lever. The real issue was that distinct cognitive jobs were being crammed into a single undifferentiated pass. The solution was to stage them separately: an adversarial review pass with frozen inputs to force self-falsification, a subtractive-editing lens to catch additive-only critique, and a secondary-thread tracker to keep sections aligned with the reader's current question. The five-dimension rubric barely changed; what changed was how reasoning was sequenced into distinct passes.
The broader takeaway for engineers building AI review or critique tools: when a reviewer plateaus after a sequencing fix, the next failures are often not rubric gaps but competing cognitive tasks sharing one pass. Naming those tasks precisely and staging them as separate observational steps can matter more than adding scoring dimensions.