DevTime scans Cal.com and admits uncertainty instead of guessing

A developer tested DevTime, a local-first CLI that scans repositories for evidence-based behavior signals, against the open-source Cal.com scheduling platform. The tool links claims about concepts like authentication, billing webhooks, and background jobs to actual code evidence, and reports uncertainty when proof is insufficient. Despite Cal.com containing a file literally named stripe/webhook.ts, DevTime flagged billing webhook behavior as low-confidence. Manual inspection confirmed the file was a stub returning a 404, with real webhook processing reserved for the enterprise edition — meaning any tool or AI agent inferring behavior from file names alone would have been wrong.

The scan wasn't flawless: it under-detected a genuine background job system (a Tasker package with Redis-backed queues and cron routes) and missed admin permission surfaces entirely. Both misses are being converted into regression fixtures to prevent silent recurrence, and the tool proactively disclosed its own blind spot around NestJS controller parsing during the run. The episode illustrates a broader concern for engineers relying on AI coding agents: plausible narratives built from file paths or dependencies can be confidently wrong, and treating uncertainty as a valid output — rather than forcing false confidence — is critical when agents might act on unverified assumptions about payment or security logic.