Scovant Blog

The token cost of a web page: how page bloat degrades AI agents

When a human visits a bloated page, they scroll past the noise and find the button they need. When an AI agent visits the same page, it has to process every token before it can act. That difference matters more than most developers expect.

This article explains how page bloat affects AI agents mechanically, what kinds of bloat are the worst offenders, and how to think about page weight in an agentic context rather than a purely human one.


What an AI agent actually does with your page

An AI agent doesn't render your page visually. It reads a representation of it — typically a cleaned or serialized version of the DOM — and reasons over that text to decide what to do next. Every element on the page becomes tokens. Every token costs attention, latency, and in some architectures, literal compute budget. You can see your own per-page token count in any Scovant scan, including the free anonymous check on the homepage.

When an agent is trying to complete a purchase or a booking, it needs to find the right form fields, understand the labels, and take the correct action. If the page surrounding that task is dense with navigation menus, cookie banners, promotional copy, modal overlays, and irrelevant structured data, the agent has to work through all of it first.

Some agents will get there anyway. Others will hallucinate a field label, misread a button state, or simply run out of context before they reach the call to action. The failure mode is silent from the user's perspective and hard to reproduce.


What counts as bloat in an agentic context

Bloat for agents is not the same as bloat for page speed. A large image adds bytes but no tokens. A massive inline SVG icon set, on the other hand, adds thousands of tokens with zero informational value to an agent trying to check out.

Here are the categories that cause the most token overhead:

1. Navigation and footer repetition

Global navigation and footers are duplicated on every page. For a human, that's invisible after the first visit. For an agent crawling a site or operating across multiple steps of a flow, that repeated boilerplate consumes token budget on every page load. If your navigation contains dozens of links, categories, and sub-menus, that structure is serialized in full every single time.

2. Cookie and consent banners

Consent banners are often among the first elements in the DOM. Agents encounter them before they reach the actual content. A verbose consent interface — with paragraph-length explanations and multiple accept tiers — can front-load hundreds of tokens of noise before the agent sees the page it came for.

3. Inline SVGs and icon markup

Designers often inline SVGs for performance reasons: no extra HTTP requests, instant render. That's a reasonable trade-off for humans. For agents, an inlined SVG for a small decorative icon can be hundreds of tokens of path data that communicates nothing. Multiply that by dozens of icons per page and you have a significant token overhead from purely decorative elements.

4. Redundant or verbose structured data

Structured data is intended to help machines understand page content. But poorly maintained schema markup can be the opposite: duplicate @context declarations, verbose nested objects, or schema types that don't match the actual page content. Agents that try to reconcile conflicting or redundant structured data with the visible page content have to do extra reasoning work.

5. A/B test and personalization scaffolding

Many sites run A/B tests or personalization layers that leave hidden variants in the DOM. The winning variant is visible, but the losing variants — with their buttons, headings, and copy — are still present in the markup, just hidden with CSS. An agent reading the DOM sees all of them. This creates ambiguity: which button is the real one? Which price is current?

6. Comment nodes and debug artifacts

Development comments, template debug output, and CMS artifacts left in production markup are invisible to humans but fully present in the token stream. These are low-priority fixes individually, but across a large site they add up.


Why this is a regression problem, not a launch problem

Page bloat tends to grow incrementally. A banner gets added for a campaign. A new analytics script injects extra DOM nodes. A design update swaps sprite sheets for inline SVGs. Each change is small. The cumulative effect is not.

This is exactly why agent-readiness needs to be treated as a regression, not a one-time audit. A page that was clean at launch can become agent-hostile over months of routine updates. Without a baseline to compare against, you won't notice until an agent starts failing flows you thought were working.

Scovant's CI regression detection compares every deploy against a baseline, which means changes that degrade agent performance get caught before they reach production. That includes changes that inflate token overhead in ways that don't show up in visual diffs or Lighthouse scores.


Why a single model test isn't enough

Different LLM agents handle noisy pages differently. A large, capable model might successfully filter out the icon SVG noise, reason over conflicting A/B test variants, and still complete the checkout flow. A smaller or more cost-optimized model running the same task on the same page might not.

If you're only testing against one model, you're testing your site's agent-readiness against the most forgiving interpreter available. The agents your users deploy are not guaranteed to be that forgiving.

Scovant runs the same flow against every LLM agent in its catalog — Claude Sonnet 4.6 is the default — so readiness reflects a distribution, not one lucky data point. The goal is to surface the cases where a page works for premium models but fails for cheaper or more constrained ones — because those failure cases are real, and they're the ones you're least likely to catch on your own.


Practical steps to reduce agent token overhead

You don't need to redesign your site. Most of the highest-value changes are surgical.

Externalize SVG icons. Reference icon files via <img> or <use> with an external sprite sheet instead of inlining full path data on every page.

Audit your consent layer. Make the consent banner markup as concise as possible. Move verbose legal copy to a linked document rather than inlining it in the DOM.

Clean up hidden A/B variants. After a test concludes, remove the losing variant from the markup entirely. Don't leave dead code in the DOM.

Simplify navigation serialization. Consider whether your mega-menu needs to be present in full in the DOM at all times, or whether it can be loaded on demand.

Validate your structured data. Use a single, accurate schema block per page type. Remove duplicate or contradictory declarations.

Remove production debug artifacts. Run a periodic sweep for HTML comments, template variables, and CMS debug output that shouldn't be in production markup.

None of these changes require a redesign or a major refactor. Most can be done incrementally, and the impact on agent performance can be measured directly. Re-scan after each change: the token count and the main-content ratio are recorded per page and per scan, so you can see the delta rather than guess at it.


Scores mostly won't show you this — ours does

Static readiness checkers can tell you whether your structured data is present and whether your robots.txt is correctly formatted. Most of them can't go further. Scovant measures the token cost of every page it crawls and raises a finding when a page's token count crosses the calibrated thresholds, alongside a main-content ratio that separates the page's substance from its chrome. What it still can't tell you on its own is whether a real agent gets confused by your inlined icon set halfway through a form — that is what the simulation is for.

That gap is what simulation-based testing is for. Scovant runs real AI agents through actual purchase and booking flows on your site, so you see what happens rather than what's declared. Combined with multi-model testing and CI regression detection, that gives you a feedback loop that reflects how agents actually experience your pages.

If you want to see where your site stands, start at scovant.com. A free check runs from the box there — one per visitor per day, up to 10 pages, no account. Scoring methodology is public at scovant.com/scoring and docs are at scovant.com/docs.

← All posts