The article discusses Google’s John Mueller’s response on Reddit about the robots meta tag and the misconception surrounding it. The robots meta tag communicates directives to search engine crawlers, instructing them on how to index content and follow links. Mueller explained that the “index” and “follow” directives have no function in Google and are completely ignored. This is because indexing and following links are the default functions of search engine robots, and they do not need to be specified. Google’s documentation on robots tags confirms this, advising that the default values are index, follow, and do not need to be specified.
The article also addresses the misconception that using the “index, follow” meta tag is beneficial. Both Google and Bing ignore this meta tag, with Bing allowing the explicit use of “index” and “follow” if necessary. The author of the article, with over 20 years of experience in SEO, advocates for giving search engine bots what they expect and removing anything that may go against the goal of making it easier for search engines to index and understand content.
The article also points out the misconception regarding the “follow” directive when used in conjunction with “noindex.” Some publishers use the “noindex, follow” meta tag believing that it compels search engines to follow the links on a page despite not being indexed. However, the article clarifies that if a page is not indexed, the links on those pages are also not indexed, and therefore cannot be followed by search engines.
In summary, the article highlights the misconception surrounding the robots meta tag, particularly the “index, follow” directive, and clarifies that both Google and Bing ignore this meta tag. The author emphasizes the importance of understanding and using the appropriate meta tags in SEO to ensure that search engine bots can effectively index and understand website content. The article provides valuable insights for publishers and SEO professionals to accurately implement robots meta tags in their websites.
Read Full Article