Auditing My AI's Slack Access Revealed a Safety Check I Never Used

An engineer granted an AI agent direct posting access to a team Slack workspace via MCP tools, then layered three safety checks on top: a name-based channel search to avoid blind use of machine-readable channel IDs, a draft-and-confirm tool for high-stakes messages, and a post-send step to reopen the channel and verify the outcome. Auditing session logs later revealed that the draft-confirm tool — adopted specifically to prevent misfired sends — had zero logged invocations across every project examined.

The audit also surfaced a smaller but telling error: an earlier account of a channel-search trace claimed the AI had located the target channel, when raw logs showed it hadn't — only an unrelated, archived channel from 2019 turned up, and the real target was never found in that sequence. The mistake was caught only when someone re-read the raw tool outputs instead of trusting a summary of them.

The broader lesson extends well beyond Slack: 'the AI's action worked' and 'I can verify what it did' are separate claims. Pre-send checks (like name search or draft confirmation) close a specification gap — did the AI understand the intent — while post-send checks close a verification gap — can the actual outcome be confirmed independently of the AI's own report. Systems that implement only one half can produce agents that are confidently wrong without anyone noticing, a failure mode already documented in real-world incidents of agents messaging the wrong destination.

For engineers building AI agents with tool access, the piece is a concrete argument for auditing actual tool-invocation logs rather than assuming safety mechanisms are being used just because they exist in the code.