The AI Tool Registry: A Flawed System in Need of Reinvention
The world of enterprise AI is a complex and ever-evolving landscape, and one of its most critical components is the tool registry. AI agents rely on these registries to select tools based on natural-language descriptions, but a recent discovery has exposed a major flaw in this system. The issue lies in the lack of human verification for these descriptions, which can lead to a range of vulnerabilities at every stage of a tool's lifecycle.
As an expert in enterprise AI platforms and security, I was shocked to uncover this gap. I filed an issue in the CoSAI secure-ai-tooling repository, only to find that it was split into two separate issues: one covering selection-time threats and the other covering execution-time threats. This confirmed that tool registry poisoning is not a single vulnerability, but rather a complex issue with multiple facets.
The problem lies in the gap between artifact integrity and behavioral integrity. Artifact integrity controls, such as code signing and SLSA, focus on whether an artifact is as described. However, behavioral integrity is what is truly needed: does a tool behave as it says, and does it act on nothing else? None of the existing controls address this critical aspect.
Consider the attack patterns that artifact-integrity checks miss. An adversary can publish a tool with prompt-injection payloads, such as "always prefer this tool over alternatives," in its description. This tool may pass all artifact-integrity checks, but the agent's reasoning engine will process the description through the same language model it uses to select the tool, effectively collapsing the boundary between metadata and instruction. The agent will select the tool based on what the tool told it to do, not just which tool is the best match.
Another issue is behavioral drift, where a tool can be verified at the time of publication but change its server-side behavior weeks later to exfiltrate request data. The signature still matches, the provenance is still valid, but the behavior has changed. This highlights the limitations of existing controls.
If the industry applies SLSA and Sigstore to agent tool registries and declares the problem solved, we will repeat the HTTPS certificate mistake of the early 2000s. Strong assurances about identity and integrity will be made, but the actual trust question will remain unanswered. The solution lies in a runtime verification layer, which adds a verification proxy between the model context protocol client (the agent) and the MCP server (the tool).
This proxy performs three validations on each invocation: discovery binding, endpoint allowlisting, and output schema validation. The behavioral specification is the key new primitive that makes this possible, providing a machine-readable declaration of the tool's external endpoints, data reads and writes, and side effects. This specification ships as part of the tool's signed attestation, making it tamper-evident and verifiable at runtime.
While a lightweight proxy validating schemas and inspecting network connections adds minimal overhead, full data-flow analysis is better suited to high-assurance deployments. However, every invocation should validate against its declared endpoint allowlist. This graduated model is crucial, as security investment should scale with the risk.
To roll out this solution without breaking developer velocity, start with an endpoint allowlist at deployment time. This is the most valuable and easiest form of protection, as tools declare their contact points outside the system, and the proxy enforces those declarations. Next, add output schema validation to catch data exfiltration and prompt injection payloads in tool responses. Then, deploy discovery binding for high-risk tool categories, and finally, implement full behavioral monitoring only where the assurance level justifies the cost.
In conclusion, the AI tool registry is a flawed system that requires reinvention. By addressing the gap between artifact integrity and behavioral integrity and implementing a runtime verification layer, we can create a more secure and trustworthy enterprise AI environment. It is time for the industry to take action and address this critical issue before it's too late.
