Docs
Build with Scovant.

Run scans, simulations, and showcase from the dashboard — or drive it all from your pipeline: the REST API, CI/CD trigger, webhooks, and the MCP interface are live today. Entries marked “Soon” are on the roadmap.

API · 3 of 4

Research data & the Agent Web Index.

Scovant publishes original research data about the agent-readiness of the web, built from its own production scans.

The Agent Web Index

A monthly aggregate over Scovant's research corpus (thousands of independent, publicly reachable websites): readiness rating distribution (the frozen R1 model through 2026-10, R2 from the 2026-11-01 snapshot), declared robots.txt policy per AI crawler (Search / Agent / Training intents), observed edge behavior toward anonymous, honest, signed and self-spoofed requests (the agent-identity matrix), and MCP / WebMCP / UCP / llms.txt adoption. Methodology, caveats, and the human view live at /research.

Machine-readable dataset

Corpus-level aggregates only — never per-site rows:

GET https://scovant.com/research/agent-web-index.json   # full monthly series
GET https://scovant.com/research/latest.json            # latest snapshot
GET https://scovant.com/research/agent-web-index.csv    # flat series (CSV)

Licensed CC BY 4.0 — reuse freely with attribution to Scovant. Each snapshot records its sample_size, a category_mix composition breakdown, and the exact ruleset_version that produced it, so any published number is reproducible. latest.json returns 404 until a snapshot has been published — never an empty object a consumer could mistake for data.

The current sample size is also exposed as research_sample_size on GET /api/public/status.

Per-site research pages

Sites in the research corpus have public detail pages at /showcase/marketing/{domain} with an evidence section (declared per-bot robots verdicts, observed WAF result, protocol detections) read from the site's latest scan. The corresponding API is GET /api/public/marketing/sites/{domain} (evidence field). Corpus sites did not opt in and are not affiliated with Scovant; a site owner can request removal at any time and removed sites leave the corpus, the pages, and the index.

Additional metrics & datasets

Beyond the headline index, the same monthly snapshot carries access_stability (how consistently a site's edge blocks or allows an honest agent fetch across recent scans), agent_identity (the anonymous/honest/signed/self-spoofed agent-identity matrix), WebMCP adoption (measured, annotated_share, read_only_share — denominated on sites that actually expose it), and category_mix (the corpus's site-type composition). From the first R2 snapshot (2026-11-01), cohorts gives the 10th to 90th percentiles of Agent Readiness and Technical Readiness for each site type, with the number of sites ranked and excluded; a site type with fewer than 30 ranked sites shows its counts only. From the 2026-12-01 snapshot definition_versions names the definition version of every metric group and definitions_version the snapshot's; a change of definitions is a series break, like a change of methodology. From then on, latest-scan metrics include only sites rated under the live Agent Readiness model (excluded_old_model_n counts the sites left out); the access-stability histories are not filtered by model. UCP adoption is split into six series under protocols.ucp, read against the official UCP v2026-08-25 profile: detected_any, parseable_legacy, valid_supported_version, valid_current_version, checkout_current_namespace and permalink_declared, each with its count; a scan that predates that reading is not measured, never counted as 0. The single ucp_profile_pct series ends with the 2026-11-01 snapshot and is not continued. Two more datasets sit alongside the index: the community-contributed series at /research/community-core-index.json and .csv (built from opt-in scovant-core --contribute submissions, kept separate from the production-scan series), and the one-off Cloudflare September 2026 edge snapshots at /research/cloudflare-2026-09.json and .csv — three independent snapshots of the marketing corpus, with the same crawler identity, taken around Cloudflare's 2026-09-15 AI-bot default change. It is not a before/after comparison: each two-day window reached a different set of sites (the overlap field counts the shared domains), and the third snapshot was measured after the crawler began following redirects to the final page, so each snapshot's shares describe that snapshot only; causality is not claimed.