No Content-Signal directives in robots.txt

What it detects

robots.txt has no Content-Signal directive declaring the site's ai-train / search / ai-input usage preferences.

Why it matters

Without an explicit signal, some edge providers default new sites to blocking AI agent traffic outright; declaring a preference is the opt-in mechanism that avoids being silently shut out.

How to fix

- Open `robots.txt` at the domain root and check for a `Content-Signal:` line declaring your content-usage preferences. - Add one under the relevant `User-agent` group, e.g. `Content-Signal: ai-train=no, search=yes` (comma-separated `name=value` pairs; the defined signals are `ai-train`, `search`, and `ai-input` — see contentsignals.org for the specification). - Choose values deliberately: `search=yes` keeps you indexable, `ai-input=yes` allows retrieval/grounding at answer time, `ai-train` controls model-training use. Explicit signals matter because edge providers (Cloudflare) default new ad-supported domains to blocking AI agent traffic — a declared signal is the opt-in mechanism. - Verify: `curl <domain>/robots.txt` and confirm the `Content-Signal:` line is present, parses as comma-separated `name=value` pairs, and matches your intended policy.

References

• RFC 9309 — Robots Exclusion Protocol
• Sitemaps protocol