How LLM Watermarking Quietly Alters AI Agent Tool-Calling Behavior

Anthropic recently confirmed that future Claude models will embed an invisible text watermark, based on Google DeepMind's SynthID-Text. The move aligns with Article 50(2) of the EU AI Act, which requires machine-readable marking of AI-generated synthetic text. But because SynthID-Text changes how each next token is sampled, watermarking turns out to affect more than provenance tracking alone.

Researchers call this effect "sampling drift": the same token-selection mechanism that embeds the watermark can also change whether a model refuses a harmful request and which tool an agent calls with what arguments. Even though SynthID-Text is statistically non-distortionary, outputs generated under a fixed watermark key can still diverge from unwatermarked generations.

Experiments on BFCL v4 for tool-calling and HarmBench/JailbreakBench for refusal behavior found that watermarking reduced tool-call accuracy in six of seven tested models, with significant drops in four. More notably, the paired disagreement rate ('churn') between watermarked and unwatermarked runs was consistently far larger than the net accuracy change — in one model, verdicts flipped on nearly 17% of items even though aggregate accuracy loss was under 3 points.

The findings suggest that provenance watermarking, while designed for compliance and traceability, can introduce measurable and sometimes hidden behavioral shifts in agentic systems — a risk developers should account for when evaluating watermarked models in production.