How To Crawl Large Websites
Crawl large websites by switching to database storage mode, increasing memory and by configuring the crawl to extract the data you need.
What Is Link Score?
Find out more about our Link Score algorithm which calculates the relative importance of URLs based on the internal linking of a site.
Back to top