John Mueller Offers Surprising News Why Google Crawls Non-Existent Pages

John Mueller Offers Surprising News Why Google Crawls Non-Existent Pages

In a Webmaster Central Hangout, Google’s John Mueller discussed the reasons why Google crawls non-existent pages and what that meant for your crawl budget. Web publishers would like to see Google crawling existing pages. It seems a waste of time for Google to crawl pages that don’t exist. A web publisher asked whether they should block Googlebot from crawling non-existent pages. John Mueller’s answer adds more information to what we know about Google’s 404 crawls.

Non-existent pages are referred to as 404 pages. That’s the server error code that a website should give when a requested web page is missing.  A 404 error code means that the server…

Read Full Article