Crawling & technical audit
How this is scored
Site crawling at real scale: what the crawler finds, how deeply it checks, whether Core Web Vitals and indexation problems are diagnosed rather than merely listed.
0 — No site crawler; the product is keyword data only.
3 — A crawler with a low page cap that reports broken links and missing tags, with no prioritisation and no rendering of JavaScript.
5 — Scheduled crawls at a stated page limit, JavaScript rendering, issue categories with severity, and Core Web Vitals pulled per template.
8 — Log-file or Search Console integration to compare what the crawler sees against what Google fetched, indexation and canonical diagnosis, structured-data validation, and issues prioritised by traffic impact rather than count.
10 — The audit is diagnostic rather than a list: crawl budget analysed, rendering differences surfaced, regressions detected between crawls with the deploy that caused them inferrable, and remediation guidance specific enough for a developer ticket.
The Skeptic
The crawl story is stated in numbers I can hold vendors to: millions of pages, up to 450 URLs per second unrendered and 350 rendered, rendering through Google's own engine, and Core Web Vitals methodology documented down to which metrics are Lighthouse-only versus CrUX origin-level. Log-file crawl-budget breakdowns, Search Console funnel reports and source-gap charts compare what the crawler sees against what Google actually fetched, issues are cross-referenced with visits and click bands rather than counted, and build gates plus an AI ticket writer turn findings into developer work. What keeps it off the top is that the captured pages show no stated page limits or crawl scheduling terms, and no report that surfaces rendering differences as such. 1 2 3 9