Accept Headers for AI Agent APIs

An AI agent should not need a custom endpoint for every page it reads. HTTP already has a mechanism for choosing a response format: the Accept header. Applying it well makes an API more useful to browsers, automation, and language models without fragmenting the product.

The two representations

For a browser, text/html is the right response. It contains the page structure, accessible controls, styles, and links needed for a person to navigate.

For an agent, text/markdown is often better. It should contain short, crisp information without navigation chrome, link lists, or extra surrounding copy. The response is easier to retrieve, reason over, and cite in a workflow.

Accept: text/html

returns the HTML document.

Accept: text/markdown

returns the same useful content as Markdown.

Make negotiation explicit

Always set Vary: Accept. This tells caches that HTML and Markdown are distinct responses. Return 406 Not Acceptable only when a client requests a format you cannot provide; otherwise serve a well-defined default.

Do not infer an agent solely from the user agent string. User-agent detection is fragile and makes debugging difficult. The requested media type is a clearer contract.

Useful response rules

1. Keep headings and terminology consistent across HTML and Markdown.
2. Preserve the canonical URL for both formats.
3. Put the concise answer first in the Markdown response.
4. Include code examples as fenced blocks.
5. Exclude marketing copy and unrelated links from the agent version.

A practical payoff

This approach gives your documentation a future-friendly interface. Human visitors get a polished site, while agents get clean source material that consumes fewer tokens and produces fewer extraction errors. It is a small piece of HTTP discipline that makes a product easier to integrate with.