Page token cost excessive for agents

What it detects

Pages whose rendered content is disproportionately large relative to the useful text an agent needs to extract.

Why it matters

Every extra token an agent has to read costs it money and context budget; bloated pages make a site more expensive and slower for agents to work with, and can push relevant content out of a limited context window entirely.

How to fix

- Inspect the page's rendered HTML/visible-text weight — look for repeated navigation chrome, footers, cookie banners, or long unpaginated listings that repeat on every page. - Reduce page weight and boilerplate noise: dedupe repeated nav/footer markup, paginate long listings, lazy-load non-critical sections, and strip redundant markup that carries no content. - Consider serving a lightweight Markdown alternate for agents (see `MARKDOWN_FOR_AGENTS_ABSENT`) so agents can skip the full HTML entirely, e.g. `<link rel="alternate" type="text/markdown" href="/products/widget.md">`. - Verify: diff the page's visible-text size before/after the cleanup and confirm the core content (price, description, CTA) is still intact.

References

• tiktoken (BPE tokenizer)
• schema.org
• JSON-LD 1.1 (W3C)