Source
Crawling & technical audit — found from sitemap — SE Ranking
Checked for SE Ranking on 1 Oct 2026
- Page
- https://seranking.com/website-crawler.html
- Checked
- 1 Oct 2026, 13:06 UTC
- How we may use it
- Public page, crawling permitted
Technical details
- type
- page
- http status
- 200
- content hash
- sha256:2598298fdf9433643712cb56ada8315d6699bbaeb90bab22d9b1bce88a77d1c1
- permission
- robots_ok
- screenshot
- Screenshot on file (internal exhibit, not published)
Cited by
Facts read from this source
-
Crawl limit per project Report an error
“Crawl up to 150,000 pages per project, with each 1,000 pages requiring no more than 2 minutes to analyze.”
-
Raw data and insights Report an error
“Get both raw crawling data as well as crawling insights”
-
Custom metrics and rules Report an error
“Set up a crawling schedule, choose a user agent, filter out irrelevant checks, adjust the list of metrics to include, and more.”
-
Crawled resources Report an error
“Explore a complete list of site URLs. Get insights into all pages, links, and resources housed on your website (e.g., images, CSS, JavaScript files), and get valuable metrics to assess their SEO health.”
-
Advanced filters Report an error
“Configure a set of advanced filters to find hidden issues or opportunities related to indexing, content, or any other page element.”
-
Page analysis parameters Report an error
“Choose the parameters to analyze crawled pages with, including their URL, headings, title, word count, canonical tags, redirects, internal/external links, page size, etc.”
-
Issue categories Report an error
“All issues identified in the Issue report are conveniently grouped into one of 18 categories. This includes security, crawling, internal/external links, usability, website speed, mobile optimization, and even textual content.”
-
Issue severity groups Report an error
“all issues are split apart into 4 groups relative to their severity level: Errors, Warnings, Notices, and Passed Checks.”
-
Crawl comparison Report an error
“Our site crawler allows you to compare crawl results side-by-side and see how your efforts are paying off over time.”
-
Scheduling Report an error
“The site crawler lets you set up automatic checks, select the scanning frequency, specify relevant metrics to check for, customize the report, adjust the restrictions used for website crawling, and so on.”
-
User agent selection Report an error
“To simulate how Google will explore your website, you can select a user agent for the tool to use during the crawl (e.g., Googlebot, Bingbot, etc).”
-
Health score Report an error
“Evaluate your site’s SEO health and compare crawl results over time to ensure your SEO efforts are paying off.”
-
Core web vitals Report an error
“Site-wide performance level metrics like LCP, FID, and CLS.”
-
Data studio template Report an error
“Send crawl data findings straight to Data Studio.”
-
Top issues breakdown Report an error
“The 5 most critical issues ordered by severity and frequency as well as page distribution by issue categories.”
-
Growth API credits Report an error
“API access with 300k credits”
-
Reporting connectors Report an error
“Data Studio / Matomo / GA / GSC”
7 facts read from this page are not shown because they could not be confirmed on the page as captured.