Crawl is a term describing a bot, script, or software program that visits a web page and grabs content and links from it. Once completed, the program visits the next link in the list or a link obtained from the web page it had recently visited.
#KnowYourInternet #Crawl
