llms.txt missing or invalid

What it detects

/llms.txt is absent, does not return HTTP 200, or fails to parse as valid Markdown.

Why it matters

llms.txt is the emerging convention for pointing agents at a site's key pages in a compact, agent-friendly format; its absence means agents fall back to guessing site structure from a full crawl.

How to fix

- Check whether `/llms.txt` exists at the domain root, returns HTTP 200, and parses as valid Markdown. - Create (or fix) `/llms.txt` with a site name, a one-line summary, and a linked list of key pages (products, docs, pricing, API reference) grouped under Markdown headings — see llmstxt.org for the canonical format. - Serve it as a static file or a simple route that returns `Content-Type: text/plain` or `text/markdown`. - Verify: `curl` `/llms.txt`, confirm a 200 status, and check it renders as clean Markdown (headings + link list, no broken syntax).

References

• RFC 9309 — Robots Exclusion Protocol
• Sitemaps protocol