Testing a Pyxel Config Tool Against Silent State Drift

A browser-based configuration tool for ESA's Pyxel detector-simulation framework doesn't own the definition of 'correct' — that lives in Pyxel itself, a separately versioned project. This creates a distinct failure mode: state drift, where the tool's bundled schema, bounds logic, or tutorial docs silently fall out of sync with Pyxel while all local tests keep passing.

The defense is built in three layers matched to where each piece of truth actually lives: a runtime freshness check that fetches Pyxel's live schema and diffs it against the bundled copy (with an explicit 'unknown' state when verification itself fails), a set of tests that run bounds-extraction logic against the real bundled schema file instead of only synthetic fixtures, and a CI step that installs a pinned Pyxel version, introspects its actual API, and validates every tutorial example against it.

The underlying principle is testing each invariant at the seam where it lives, rather than trusting frozen fixtures that can drift right along with the source of truth. The piece also covers the split between automated deterministic tests and manually verified rendered output, and how per-file coverage gates in CI make ongoing test coverage non-optional rather than aspirational.