How to set or enable or Add Custom Robots.txt file in Blogger

How to Add Custom Robots.txt file in Blogger

First if you have interest that what is Custom Robots.txt file? Than you Can read

Robots.txt file is a text file which have few lines of simple code. Saved on the website or blog’s server which instruct the web crawlers to how to index and crawl a blog in the search results. That means you can restrict any web page on your blog from web crawlers so that it can’t get indexed in search engines like your blog labels page, your demo page or any other pages that are not as important to get indexed. Always remember that search crawlers scan the robots.txt file before crawling any web page.

Add Custom Robots.txt file in Blogger

Adding Custom Robots.Txt to a Blogger

Step1: At First Go to your blogger blog.

Step2: Go to at the bottom and find Settings >> Search Preferences ››Crawlers and indexing ›› Custom robots.txt ›› Edit ›› Yes

Step3:Now paste your robots.txt file code in the box.

User-agent: Mediapartners-Google
Disallow:
User-agent: *
Disallow: /search
Allow: /
Sitemap: http://yourblogger.blogspot.com/feeds/posts/default?orderby=UPDATED

Step4: Now Click on Save Changes button.

step of adding custom robots txt file

Now you are able to set or Add or enable Custom Robots.txt file in Blogger

How to Check Your Robots.txt File?

You can check this file on your blog by adding /robots.txt at last to your blog URL in the browser.

http://www.yourblogger.blogspot.com/robots.txt

For My blogger it will be Look like below:

Robots.txt File

Now you are easily can set or Add or enable Custom Robots.txt file in Blogger and check the file.

Pages