Agent standards, compared
Several competing files and protocols aim to make a site or tool readable and actionable by AI agents. None is universal. This is an honest matrix — including where adoption is contested.
Rigor note. This field moves fast and some adoption is claimed but not confirmed by model providers. Where that happens, we say so. Last reviewed: 2026-06-13.
| Standard | What it is | Proposed by | Adoption | What it solves |
|---|---|---|---|---|
| llms.txt | A Markdown file at /llms.txt with a curated, LLM-friendly site map. | Jeremy Howard / Answer.AI (2024). | Growing among technical sites; contested — no major provider has confirmed consuming it in production. | Curated, readable entry point for models. |
| ai.txt | A robots-style file of AI usage rules (several competing proposals). | Multiple (e.g. Spawning for training opt-out). | Fragmented; no clear winner. | Usage/training permissions. |
| agents.txt | A robots-like file aimed at agents, or a pointer to API specs. | Scattered community proposals. | Very early; no convergence. | Rules and navigation hints for agents. |
| agent.json / Agent Card | JSON under /.well-known/ describing an agent (capabilities, endpoints), not a site. | Google, in the A2A (Agent2Agent) protocol, 2025. | Emerging in agent interoperability. | Agent-to-agent discovery. |
| MCP (Model Context Protocol) | An open protocol connecting models to tools and data via MCP servers. | Anthropic (2024); later adopted by others. | Strong and growing — the most real on this list for action. | Actionable layer: tools, resources, prompts. |
| NLWeb | Turns a site into an agent-queryable conversational interface using schema.org + LLMs; each instance is also an MCP server. | Microsoft (R.V. Guha), 2025. | Early; promising. | Natural-language queries over site content. |
Practical verdict
| Standard | Worth it today? |
|---|---|
| llms.txt | Yes, as curation. Cheap to maintain and useful as an index; do not count on guaranteed model consumption. (This site ships one.) |
| ai.txt | Optional. Until convergence, training permissions live better in robots.txt. We keep an ai.txt for interoperability. |
| agents.txt | Wait. Too early to invest. |
| agent.json | Only if you build an agent. It describes agents, not sites. Watch A2A. |
| MCP | Yes, if you have real actions. The actionable layer with serious adoption. See our API & MCP page (proposed, not yet live). |
| NLWeb | Promising, watch it. Interesting if you already use rich schema.org. |
How to apply these in sequence: see the method (Agent Entry Chain) and the playbook.