Quick Guide: "Ask Search Engines Not to Index This Site"

In our WordPress, it can appear in several ways. For example:

• Ask search engines not to index this site
• Discourage search engines from indexing this site

But what does it really mean? How can it affect our website? How does it work? Keep reading and you'll discover this and much more.

If you want to learn more about WordPress, I recommend checking out this article on free WordPress courses.

What does "Ask search engines not to index this site" mean?

If, from the WordPress dashboard, we go to Settings/Reading, we'll find a series of options to select the home page, the blog page, etc. The last option seems a little out of context. And it's normal for it to seem that way, since it has nothing to do with the previous selectors.

If this checkbox is enabled, we are telling search engines (such as Google) that this website should not appear in search results.

We'll also find this checkbox at the time of installing WordPress.

How to check whether this checkbox is enabled or disabled

Having this checkbox enabled will prevent Google from placing our website in search results. So, if we don't explicitly want this to happen, we must make sure the checkbox is disabled. To verify this we need to:

1. Log in to the WordPress dashboard
2. Click on Settings
3. Click on Reading
4. At the bottom, just above the «Save Changes» button, the checkbox should appear
5. If it has a blue check it means it is enabled

When to enable the "Ask search engines not to index this site" checkbox

If we have a website, the most logical thing is that we want to appear in search results. But in some cases we may want the opposite. I'll explain some of them below:

1. If we are working online on a new website but it isn't ready for launch
2. If we are working on a new version of our website we should have a copy (or staging environment) for testing and modifications. This version should not be indexed
3. If our website is not aimed at attracting users and we want to use it for something else, such as giving access to a private group of users
4. If we have content copied from another website and we don't want Google to treat it as duplicate. This way it won't affect the SEO of the original website

In summary, we will enable this checkbox when we want our website not to be taken into account for any search on the internet.

What happens when we enable this checkbox

When we enable the checkbox to discourage search engines, a meta tag located in the header of our website will be modified. We are talking about the robots meta tag, which will be given the value noindex.

This sends a signal to the bots that are constantly crawling websites on the internet, so that they do not index it or treat the content as indexable.

This will not prevent bots from visiting our website, but it will prevent it from being taken into account for ranking purposes.

What to keep in mind if we enable the "Ask search engines not to index this site" checkbox

It's normal to want to enable this checkbox temporarily. The only thing to keep in mind is to disable it again once we've finished working on the website. If we don't, users won't be able to find us through Google.

After disabling the "Discourage search engines..." checkbox, we'll need to do the following to make sure Google will index our website:

• Install Google Search Console
• Submit our sitemap in Google Search Console
• We can use a number of tools to try to get the site indexed sooner (Although only Google knows when that will happen…)

What other ways are there to prevent our content from being indexed

Apart from the "Ask search engines not to index this site" checkbox, there are also other ways to prevent bots, spiders, or crawlers from taking a website into account. Among others, we can find the following:

1. If we use Yoast in WordPress we can mark any page we wish as noindex
2. We can prevent indexation from the robots.txt file
3. We can directly add/modify the robots meta tag
4. We can put a password on the page to restrict access