Scovant Blog

A Passing Agent Simulation Isn't Agent-Ready: A SaaS Teardown

urlscan.io — a well-known URL-scanning and security tool with a real, documented public API — scored 87.5 / 100 for agent readiness when we scanned it in late August 2026, the top of the SaaS pack we looked at. Its robots.txt welcomes AI agents, an honest fetch isn't WAF-blocked, and its structured data is a clean 100. We pointed a real browser-agent at it, and it succeeded at its task. So this isn't a failure story. It's a subtler and more useful one: a passing simulation still doesn't mean every agent can use you.

What the simulation showed — success

We gave Claude Sonnet 4.6 a real browser and a SaaS discovery task: find the plans and the signup path. In three steps it navigated to urlscan.io, went straight to the pricing page, and reported the task done — success. A browser-driving agent doing a UI task had no trouble. That's a real, honest result, and it's the point: if you only test one kind of agent, urlscan looks completely ready.

What the scan caught that the simulation couldn't

A single passing simulation covers one class of consumer — a browser-driving agent doing a UI task. Scovant's static scan surfaces the classes it doesn't:

Structured 100 says the marketing pages are well-built. Actionability 59 says there's little here an agent can act on programmatically. The passing simulation and the scan findings aren't in conflict — they describe different agents.

The CI angle — you can only regression-test a surface you declared

This is where SaaS diverges from a checkout teardown. urlscan already ships an API; its exposure isn't "no capability," it's "no declared, machine-discoverable capability — and therefore nothing to protect on deploy."

The order is the lesson: declare the surface, then you can regression-test it. A SaaS that never declares an agent interface has nothing for CI to protect, and learns it broke agents only when they stop calling.

The fix list

  1. Serve critical content without requiring JavaScript (server render or a static fallback) so the citation layer — answer engines, non-browser crawlers — can read the product, not just a full-browser agent.
  2. Declare a machine-discoverable API/agent surface (MCP discovery or an agent-interface manifest) over the API urlscan already runs.
  3. Wire that surface into CI so an interface change is caught as a regression on deploy, not in production.
  4. Citation and discovery hygiene: OpenGraph, llms.txt, Link headers.

The honest takeaway

urlscan earns its 87, and a real agent completed a real task on it — we're not going to pretend otherwise. But "a browser-agent succeeded" is not "agents can use us." The answer engines that would cite urlscan see a JavaScript shell; the API-first agents that would call it can't discover the surface; and with nothing declared, there's nothing for CI to protect on the next deploy. That's the case for testing more than one class of agent, and for running the scan and the simulation together — each caught what the other missed.

Scovant scanned urlscan.io as part of its public agent-readiness research corpus. The site is not affiliated with or endorsed by Scovant; removal requests are honored. See the research methodology and the public rules catalog.

← All posts