Crawling & technical audit
How this is scored
Site crawling at real scale: what the crawler finds, how deeply it checks, whether Core Web Vitals and indexation problems are diagnosed rather than merely listed.
0 — No site crawler; the product is keyword data only.
3 — A crawler with a low page cap that reports broken links and missing tags, with no prioritisation and no rendering of JavaScript.
5 — Scheduled crawls at a stated page limit, JavaScript rendering, issue categories with severity, and Core Web Vitals pulled per template.
8 — Log-file or Search Console integration to compare what the crawler sees against what Google fetched, indexation and canonical diagnosis, structured-data validation, and issues prioritised by traffic impact rather than count.
10 — The audit is diagnostic rather than a list: crawl budget analysed, rendering differences surfaced, regressions detected between crawls with the deploy that caused them inferrable, and remediation guidance specific enough for a developer ticket.
The In-House SEO Lead
Weekly Rogerbot crawls run at published limits of 100,000 to 1,250,000 pages per week, issues come in categories like Critical Crawler Issues, Crawler Warnings, Redirect Issues, Metadata Issues and Content Issues with fix guidance, and Core Web Vitals are bulk-analysed per page with defined mobile and desktop bands and change tracking between crawls. Two things cap it for me: I found no public information on JavaScript rendering, Search Console or log-file comparison, canonical diagnosis or structured-data validation, and the help page itself says the crawlers are based in North America, which can impact accuracy for international sites — my German site included. 2 4 5