]> Independent

contactkaylacard@gmail.com

Applications and Real-Time Independent Submission AI agents well-known URI capability declaration capability discovery MCP Model Context Protocol A2A This document requests registration of two Well-Known URIs under the "/.well-known/" path: "agents.txt" and "agents.json". These URIs define a machine-readable capability declaration format: a positive statement of what web agents CAN do on a site -- which endpoints are sanctioned for agent use, which protocols (REST, MCP, A2A, GraphQL, WebSocket) are supported, what authentication mechanisms are expected, and what rate limits the site advertises. This is distinct from "robots.txt", which uses a restriction syntax to declare what crawlers may not do. Where "robots.txt" expresses prohibition, "agents.txt" expresses capability -- a sanctioned channel for agent interaction that is otherwise routinely blocked by bot detection, CAPTCHAs, and rate limiters because no positive declaration surface exists.

Introduction Automated AI agents increasingly interact with websites to perform tasks on behalf of users: searching product catalogs, retrieving structured data, executing transactions, and calling APIs. These agents are routinely blocked by bot detection systems, CAPTCHAs, and rate limiters because no sanctioned channel for agent interaction exists. Simultaneously, website operators have no standard mechanism to declare which agent behaviors they support, which endpoints are designed for machine access, or how agents should authenticate. "agents.txt" addresses this gap. It is an opt-in capability declaration file, served at a well-known location, that communicates to AI agents:

• What capabilities are available (search, browse, transact, etc.)
• Which protocols are supported (REST, MCP , A2A, GraphQL, WebSocket)
• What authentication mechanisms are required (and where to obtain tokens)
• What rate limits the site declares
• Which agents are permitted and under what conditions

Relationship to Existing Standards "agents.txt" is complementary to, and does not replace, existing standards:

robots.txt:

Declares crawling restrictions. "agents.txt" declares what agents are explicitly permitted to do. Both files may coexist.

llms.txt :

Provides human-readable content for LLMs to read. "agents.txt" declares machine-callable endpoints and capabilities.

security.txt :

Declares security vulnerability disclosure contacts. "agents.txt" declares AI agent interaction policies.

OpenAPI:

Documents individual API endpoints in detail. "agents.txt" is a discovery layer; it may reference OpenAPI specifications per capability.

MCP (Model Context Protocol):

A protocol for AI tools. "agents.txt" can declare MCP endpoints, making them discoverable without prior configuration.

Related Work The following efforts overlap with or are adjacent to this document. None of them provide a site-side capability declaration in the form defined here.

draft-srijal-agents-policy-00 (expired) :

A prior Internet-Draft using the "AGENTS.TXT" name. Its design is a strict policy file with path-based ALLOW/DISALLOW directives modeled on "robots.txt". It expired April 2026 with no -01 revision. The present document differs in scope (capability declaration, not policy restriction) and structure (typed capability and agent blocks with protocol and authentication metadata).

MCP server-card.json :

The Model Context Protocol defines a "/.well-known/mcp/server-card.json" file describing an individual MCP server's tools and resources. "agents.txt" operates one layer up: it declares that a site offers capabilities, including (optionally) MCP endpoints, and points to the MCP server-card for tool-level detail. The two are complementary.

A2A agent-card.json :

The Agent-to-Agent (A2A) protocol defines a "/.well-known/agent-card.json" file describing the capabilities of a single agent endpoint. "agents.txt" operates at the site level and may reference one or more A2A agent cards via Capability blocks with Protocol: A2A.

llms.txt :

A site-level Markdown file at "/llms.txt" that summarizes site content for LLM consumption. It is a content-summary surface, not a capability declaration. "agents.txt" and "llms.txt" can coexist on the same site without conflict; they answer different questions ("what content is here" vs. "what actions are sanctioned").

Requirements Language The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 when, and only when, they appear in all capitals, as shown here.

The "agents.txt" Well-Known URI

Location The "agents.txt" file MUST be served at: Agents SHOULD also check the root path as a fallback: The "/.well-known/agents.txt" path takes precedence when both exist. The file MUST be served over HTTPS in production deployments. HTTP is permitted only in development or testing environments. The file MUST be served with Content-Type "text/plain; charset=utf-8".

Format The "agents.txt" file uses a block-based key-value format inspired by "robots.txt". Each line contains a key, a colon, and a value. Lines beginning with "#" are comments. Indented lines (two or more spaces, or one or more tabs) belong to the preceding block. A minimal "agents.txt" file:

Header Fields

Spec-Version (REQUIRED):

The specification version. MUST be "1.0" for documents conforming to this specification.

Generated-At (OPTIONAL):

ISO 8601 timestamp of when the file was generated.

Declaration-Type (OPTIONAL):

One of "platform" or "agent". Default: "platform". A platform declaration states what agents may do on this site (the standard case). An agent declaration, published by an agent operator at the operator's own domain, states what the operator's agent does on external platforms.

Operates-On (OPTIONAL):

URL of a platform this agent operates on. MAY appear multiple times. Expected when Declaration-Type is "agent"; not used in platform declarations.

Site Fields

Site-Name (REQUIRED):

Human-readable name of the site or service.

Site-URL (REQUIRED):

Canonical HTTPS URL of the site.

Site-Description (OPTIONAL):

Brief description of the site.

Site-Contact (OPTIONAL):

Contact email address for agent-related inquiries.

Site-Privacy-Policy (OPTIONAL):

URL of the site's privacy policy.

Capability Blocks A Capability block declares a single action available to agents. Capability identifiers MUST consist of lowercase letters, digits, and hyphens only.

Capability (REQUIRED):

Identifier for this capability.

Endpoint (REQUIRED):

Full HTTPS URL of the endpoint.

Protocol (REQUIRED):

The interaction protocol. One of: REST, MCP, A2A, GraphQL, WebSocket.

Method (OPTIONAL):

HTTP method for REST endpoints. Default: GET.

Auth (OPTIONAL):

Authentication type. One of: none, api-key, bearer-token, oauth2, hmac. Default: none. Servers MUST NOT include actual credentials in this field.

Auth-Endpoint (OPTIONAL):

URL where agents obtain authentication tokens. MUST be present when Auth is "bearer-token" or "oauth2".

Auth-Docs (OPTIONAL):

URL of human-readable documentation describing the authentication flow for this capability.

Scopes (OPTIONAL):

Comma-separated list of OAuth2 scopes required by this capability.

Rate-Limit (OPTIONAL):

Advisory rate limit in the format "N/window" where window is one of: second, minute, hour, day. Agents SHOULD respect declared limits. Servers MUST enforce limits independently.

Description (OPTIONAL):

Human-readable description of the capability.

OpenAPI (OPTIONAL):

URL to an OpenAPI specification document describing the endpoint.

Param (OPTIONAL):

Declares one parameter of a REST endpoint. MAY appear multiple times within a Capability block. The value uses the form:

: where "location" is one of "query", "path", "header", or "body"; "type" is one of "string", "integer", "number", or "boolean"; the literal token "required", when present, marks the parameter as required; and the free-text description after "-" is optional. Example: Fields not defined in this document MUST be ignored by parsers, to permit forward-compatible extension.

Access Control Fields

Allow (OPTIONAL):

Glob pattern for paths agents may access. Semantics follow "robots.txt" conventions.

Disallow (OPTIONAL):

Glob pattern for paths agents MUST NOT access.

More specific patterns take precedence over less specific patterns. When no access control is declared, only paths referenced by capabilities are implicitly permitted.

Agent Policy Blocks Agent blocks declare per-agent policies. The wildcard "*" declares the default policy for all agents. Agent identifiers SHOULD match the first token of the agent's User-Agent header (case-insensitive).

Capabilities (OPTIONAL within an Agent block):

Comma-separated list of capability identifiers this agent is permitted to use. If omitted, all declared capabilities are permitted.

Agent-Declaration (OPTIONAL within an Agent block):

URL of the agent operator's own "agents.txt" file (a declaration with Declaration-Type "agent"). Enables cross-referencing between a platform's grant of capabilities and the agent operator's published statement of what the agent does and where it operates.

The "agents.json" Well-Known URI

Location The JSON companion file MUST be served at: The file MUST be served with Content-Type "application/json; charset=utf-8".

Format The JSON format contains equivalent information to "agents.txt" in a typed JSON structure suitable for direct consumption by programmatic clients. The "agents.txt" file MAY reference the JSON file via: A minimal "agents.json" document: Field semantics are identical to those defined in Section 2 for the text format.

Agent Behavior

Discovery Agents SHOULD fetch "/.well-known/agents.txt" and/or "/.well-known/agents.json" before interacting with an unfamiliar site. Agents SHOULD prefer the JSON format when both are available, as it is more precisely typed and unambiguous. Agents SHOULD cache the capability declaration for the duration declared by the HTTP Cache-Control header. Implementations SHOULD use a minimum cache TTL of 60 seconds to reduce server load.

Identification Agents SHOULD identify themselves via the User-Agent HTTP header when calling capability endpoints. The agent name in the User-Agent header is matched (case-insensitively) against Agent blocks to apply per-agent policies.

Rate Limiting Agents SHOULD respect Rate-Limit declarations as advisory limits. Servers MUST enforce rate limits independently and MUST NOT rely on agents to self-enforce.

Server Behavior

Caching Servers SHOULD serve "agents.txt" and "agents.json" with appropriate Cache-Control headers. A max-age of 300 seconds (5 minutes) is RECOMMENDED for most deployments.

CORS Servers SHOULD include the following headers to permit cross-origin discovery:

Security Considerations Capability declarations MUST NOT include actual credentials, API keys, tokens, or secrets of any kind. The Auth and Auth-Endpoint fields describe mechanisms only. Servers MUST enforce all declared restrictions (rate limits, access control, agent policies) independently of the declarations in "agents.txt". The file is advisory to agents; it is not a trust boundary. Agents MUST validate that capability endpoints use HTTPS before sending authentication credentials. Site owners SHOULD review their capability declarations periodically to ensure they accurately reflect current server capabilities and access policies.

IANA Considerations

Well-Known URI Registration: "agents.txt" This document requests registration of the following Well-Known URI in the "Well-Known URIs" registry established by :

URI suffix:

agents.txt

Change controller:

Kayla Cardillo

Specification document(s):

This document.

Related information:

Text-format capability declaration file for AI agent discovery.

Well-Known URI Registration: "agents.json"

URI suffix:

agents.json

Change controller:

Kayla Cardillo

Specification document(s):

This document.

Related information:

JSON-format capability declaration file for AI agent discovery. Companion format to agents.txt.

References Normative References In many standards track documents several words are used to signify the requirements in the specification. These words are often capitalized. This document defines these words as they should be interpreted in IETF documents. This document specifies an Internet Best Current Practices for the Internet Community, and requests discussion and suggestions for improvements. RFC 2119 specifies common key words that may be used in protocol specifications. This document aims to reduce the ambiguity by clarifying that only UPPERCASE usage of the key words have the defined special meanings. This memo defines a path prefix for "well-known locations", "/.well-known/", in selected Uniform Resource Identifier (URI) schemes. In doing so, it obsoletes RFC 5785 and updates the URI schemes defined in RFC 7230 to reserve that space. It also updates RFC 7595 to track URI schemes that support well-known URIs in their registry. The Hypertext Transfer Protocol (HTTP) is a stateless application-level protocol for distributed, collaborative, hypertext information systems. This document describes the overall architecture of HTTP, establishes common terminology, and defines aspects of the protocol that are shared by all versions. In this definition are core protocol elements, extensibility mechanisms, and the "http" and "https" Uniform Resource Identifier (URI) schemes. This document updates RFC 3864 and obsoletes RFCs 2818, 7231, 7232, 7233, 7235, 7538, 7615, 7694, and portions of 7230. Informative References

Example: E-Commerce Site

Acknowledgments The "agents.txt" format draws on the design of "robots.txt" , "security.txt" , and OpenAPI for structural inspiration. The MCP protocol reference is to the Model Context Protocol specification.