Refactyl's verifier had more bugs than the code it checked

Refactyl, a tool that automates codebase migrations like Vue 2 to Vue 3, validates every converted component against Vue's own compiler before shipping. During a live migration, though, entirely valid Vue 3 output started getting rejected by that very gate. The root cause was a performance shortcut: the gate reused the AST produced by parse() and fed it directly into compileTemplate(), which silently assumes metadata that AST doesn't always carry — especially for certain v-if and slot structures — causing codegen to fail on perfectly valid code.

What made this dangerous was its invisibility. The pipeline was designed to fail safely: reject a file, keep the original, flag it, move on. Nothing crashed, nothing errored. The only visible symptom was a dip in conversion rate and a rise in flags — metrics that looked like appropriate caution, not a defect quietly discarding good work. The fix was to run both parsing stages directly from source text, never passing a pre-parsed AST across the boundary.

The broader lesson applies to any validation pipeline — CI, linters, schema checks, LLM output filters. False passes are loud and self-correcting because users hit them and file bugs. False fails get silently absorbed by the very safety mechanisms meant to make failure harmless. Refactyl's team now tests verifiers as production code, monitors fallback paths with the same rigor as error paths, and insists that gates check the exact artifact that ships — not a convenient intermediate representation.