Site overview
Overview is the first screen when you open a site, and it answers a single question: what state is this site in right now? It gathers the headline stats, the open warnings and the full optimization dashboard.
Stat cards#
| Card | What it counts | What to look at |
|---|---|---|
| Total Pages | Pages discovered by the crawler. | Compare it with the real size of your site. A far smaller number points at an incomplete sitemap. |
| Optimized | Pages already rewritten for AI agents. | This is the number that matters: only these reach the models. |
| Errors | Pages that failed to scrape or to optimize. | Should be 0. If it is not, use Retry all from Pages. |
| Sitemap | Whether a sitemap.xml was found. | Not found severely limits page discovery. |
Page Status Distribution#
A stacked bar splitting every page on the site by state: Optimized, Scraped, Pending, In progress and Errors, with the count for each segment.
The usual reading is a green stretch (optimized) followed by a large orange stretch (scraped). That orange is pending work: pages whose content is already stored but which AI agents still cannot see.
Quick Actions#
Crawl Information#
Shows the date of the last crawl (or Never crawled) and the max pages configured for this site.
Configuration Issues#
When AgentFi detects something that stops AI agents from reading your site properly, it shows it here and in a strip under the header. Every warning offers:
- Show recommendations — expands the concrete steps to fix it.
- Re-check now — runs the check again. If the problem is still there the panel says so; if it is gone, the warning disappears.
A typical warning is the cookie redirect: if your site answers with a 302 and sets cookies before serving content, AgentFi's own crawler copes with it, but external agents (GPTBot, ClaudeBot, PerplexityBot) will receive the home page instead of the page they asked for.
Some warning types cannot be re-checked automatically; when that happens the panel says so explicitly.
Auto-optimization bar#
Right below sits the state of auto-optimization: active, paused or off, with the date of the last run and a Set up or Manage button. See Auto-optimization.
Optimization Dashboard#
The second half of the Overview gathers, in a single view, the indicators that also exist as standalone sections.
Discovery Files#
The state of sitemap.xml, llms.txt and llms-full.txt: whether they were found on your site and whether AgentFi manages them or uses the originals. Details in Discovery files.
Page Coverage#
Repeats the counts of total, optimized, pending and error pages, and adds the two coverage metrics:
- Scrape coverage — how much of the site has stored content.
- Optimization coverage — how much of it is ready for AI agents.
Content Quality#
Bot traffic and referral traffic#
7- and 30-day summaries of crawler visits and of human referrals from AI platforms, with their charts and tables. The full detail is in Bot traffic and LLM referral traffic.