Source
Crawling & technical audit — found from sitemap — AgencyAnalytics
Checked for AgencyAnalytics on 11 Nov 2026
- Page
- https://help.agencyanalytics.com/en/articles/2146439-site-auditor-faq
- Checked
- 11 Nov 2026, 00:35 UTC
- How we may use it
- Public page, crawling permitted
Technical details
- type
- page
- http status
- 200
- content hash
- sha256:98d69c4b2e7d4fcc2b4a4d64485c6331ad813b3a75543f8031487dd8371a6fe7
- permission
- robots_ok
- screenshot
- Screenshot on file (internal exhibit, not published)
Cited by
Facts read from this source
-
Site auditor paid addon Report an error
“Site Auditor is available as a paid add-on for accounts created before November 2023.”
-
Crawl inclusions Report an error
“All links (<a href>) found on any page, including links that are visually hidden”
-
Crawl skips Report an error
“The only pages it skips are those blocked via robots.txt, added to the Excluded URLs / Paths setting, or using a non-HTTP/HTTPS URL.”
-
URL dedup Report an error
“HTTP, HTTPS, www, and non-www versions of a URL are treated as the same page.”
-
Root noindex stops crawl Report an error
“If the client's root URL returns an X-Robots-Tag: noindex HTTP header, the crawl stops immediately and returns 0 pages.”
-
Exclude rule Report an error
“Paths are excluded using a contains rule, so anything matching the path you enter is skipped.”
-
No wildcards regex Report an error
“Not yet. The exclusions box doesn't currently accept wildcards or regular expressions, but this is on our roadmap and will likely be released at a future date.”
-
Ignore behavior Report an error
“Ignoring a test or page still causes it to be crawled, but hides it from your results and removes it from your audit score.”
-
Ignored never scored Report an error
“No. Whether ignored issues are shown or hidden, they're never included in your site audit scores.”
-
Page details tabs Report an error
“The Page Details slide-out has the following tabs:”
-
Insights metrics Report an error
“It includes crawl metadata like the date it was last scanned, status code, and load time, plus on-page metrics like title length, page size, URL length, number of internal and external links, and total image count.”
-
Audit history Report an error
“Site Auditor keeps your past audits so you can review earlier pages and tests.”
-
Audit compare Report an error
“Once Compare To is enabled, a change percentage indicator appears in the table widget on your dashboard, showing how results have changed between the two audits.”
-
Export formats Report an error
“You can export your audit as a PDF or an XLS file, or share it via a link.”
-
Crawler user agent Report an error
“You can allow or block our crawler specifically using the user agent "Mozilla/5.0 (compatible; RSiteAuditor)" in your robots.txt file.”
-
Crawler ipv4 Report an error
“If the auditor still can't connect, whitelist the following IP addresses on your website or server firewall.”
-
Crawler ipv6 Report an error
“2a01:4f8:c17:f386::1/128”
-
Duplicate content algorithm Report an error
“Each flagged page is given a similarity score from 0 to 10.”
-
Word count scope Report an error
“our Site Auditor counts visible and hidden text (including text hidden by CSS, like hover menus), but excludes content that isn't considered part of the page document, such as video fallback text.”
-
Page limit reset Report an error
“Your Site Audit page limit resets at midnight UTC, at the start of each billing cycle.”
-
Security block threshold Report an error
“When at least 5 pages, and at least 10% of the pages crawled, are refused, the Audit Error column in the Audit History table says so”
-
Crawl rate Report an error
“The crawler already waits 5 seconds between pages.”
-
Crawl limit max Report an error
“Yes, you can set a crawl limit between 100 and 30,000 pages per audit in the client's Site Auditor settings.”
-
Insufficient credits cause Report an error
“This error means your crawl limit for that client is set higher than the pages remaining in your account's monthly allowance.”
-
Hours Report an error
“We're available 24/5 to help”
1 fact read from this page is not shown because it could not be confirmed on the page as captured.
Scores citing this record
- The Agency Owner Crawling & technical audit
- The Agency Owner Pricing transparency
- The Content Strategist Crawling & technical audit
- The Data Analyst Crawling & technical audit
- The Data Analyst Pricing transparency
- The In-House SEO Lead Crawling & technical audit
- The In-House SEO Lead Pricing transparency
- The Skeptic Crawling & technical audit
- The Skeptic Pricing transparency
- The Technical SEO Crawling & technical audit