First Large-Scale Study Finds Credential Leaks in LLM Agent Skills

A new empirical study sampled 17,022 skills from SkillsMP, the largest open-source marketplace for LLM agent skills, combining static secret extraction, dynamic sandbox testing, and cross-referencing of developer intent against runtime behavior. The analysis identified 520 affected skills containing 1,708 credential-related security issues, organized into a taxonomy of 10 distinct leakage patterns.

The results show that 76.3% of cases require analyzing natural-language descriptions and code together, since a skill's real behavior often diverges from its documented intent. Debug logging alone accounts for 73.5% of vulnerabilities, because agent frameworks feed stdout output directly into the LLM's context window, turning routine debug statements into credential exposure vectors.

Most leaked credentials (89.6%) are immediately exploitable, and 92.5% require no elevated privileges during normal execution. The fork-based distribution model common on skill marketplaces undermines remediation efforts: secrets removed from 107 upstream repositories persist across more than 50 independent forks. Following responsible disclosure, malicious skills were removed and 91.6% of hardcoded credential cases were fixed; the researchers released their dataset, taxonomy, and detection pipeline for future agent security research.