Rethinking MCP Security: A Large-Scale Study of 64K Runtime Servers
MCPZoo tests 64,611 MCP servers, revealing that most security scanner alerts are false positives and scanners disagree widely.
Researchers introduced MCPZoo, the largest dynamic-analysis collection to date for the Model Context Protocol (MCP), the emerging standard letting LLM-based agents call external tools. A multi-agent framework converts static, in-the-wild repositories into runnable services by combining environment inference with iterative, feedback-driven repair, yielding 64,611 unique MCP servers, over 37,288 of which support full dynamic analysis.
Using this dataset, the team ran the first ecosystem-scale measurement of MCP servers alongside the security scanners meant to assess them. While existing scanners flag 96.89% of servers as risky, manual validation found fewer than half of sampled alerts were true positives, with substantial disagreement between different scanners on the same servers.
For engineers building or auditing MCP integrations, the findings suggest current automated scanning results should be treated with caution rather than as ground truth. The team has also released a public query interface to support more reliable, reproducible risk assessment of MCP servers.