Crawling & technical audit — found from sitemap — Netpeak Spider

Cited by

Facts read from this source

• Crawling rules purpose Report an error

  “Crawling rules specify which type of URLs to include or exclude from crawling.”

• Rules unlimited Report an error

  “Add a rule → adds new rules in the settings window. It is also possible to add a rule using the Ctrl+N hotkey. The number of rules is not limited.”

• Rule filter logic Report an error

  “AND → combines two or more rules: it returns ‘true‘ only if all of setrules return ‘true‘. OR → returns ‘true‘ if at least one of set rules returns ‘true‘.”

• Rule actions Report an error

  “Include → the crawler will add URLs matching the conditions, into the table. Exclude → URLmatching the conditions, will be included to the ‘Skipped URLs‘ table.”

• Rule conditions Report an error

  “Contains → restrict crawling by the text content in a URL, for example, of the specified category . Exactly matching → for searching or excluding a particular URL from the report. Matching RegExp → allows to include/exclude URLs using regular expressions.”

• Follow links on filtered urls Report an error

  “‘Follow the links on filtered URLs‘ → this setting is similar to the tag. Check it to follow all links located on the pages that meet the rules.”

• Rules combinable with other settings Report an error

  “Netpeak Spider allows combining described conditions between each other and with other settings.”

• Filtered urls report Report an error

  “Note that filtered URLs will not be added to the results table but will be included in the ‘Skipped URLs‘ report.”

Scores citing this record

• The Content Strategist Crawling & technical audit
• The Data Analyst Crawling & technical audit
• The In-House SEO Lead Crawling & technical audit
• The Technical SEO Crawling & technical audit