How to index blog category on Blogger Website

  How to index blog category on Blogger Website

                If you are unable to index a category on your Blogger website and you keep getting a message that it's blocked by robots.txt, it means that the category page has been blocked from search engines in your website's robots.txt file.



What is robots.txt file?

    The robots.txt file is a standard used by websites to communicate with web crawlers and other automated agents like search engine bots. The file specifies which parts of the website can be crawled and indexed by search engines and which parts should be blocked.

 To allow search engines to index the blocked category page on your Blogger website, you will need to modify your website's robots.txt file. Here are the steps to do so:

4 Steps to modify Blogger website's robots.txt file

Step 1: 

Go to your Blogger dashboard and click on "Settings".

Step 2: 

Click on "Search preferences" and then click on "Edit" next to "Custom robots.txt".

Step 3:

 In the "Custom robots.txt" section, make sure that the checkbox next to "Enable custom robots.txt content?" is checked.

Step 4:

 In the "Content" section, add the following lines of code to allow search engines to crawl and index the blocked category page:

User-agent: *

Disallow:

Disallow: /search/label/yourcategoryname

Replace "yourcategoryname" with the name of the category that you want to allow search engines to crawl and index.

Step 5: 

Click on "Save changes" to save the modified robots.txt file.

It may take some time for search engines to recrawl your website and index the previously blocked category page. So, be patient and keep checking your website's search engine rankings over time.

User-agent: *

Disallow:

Disallow: /search/label/

    This code will allow search engine crawlers to crawl and index all the category pages on your website. The Disallow: /search/label/ line tells the search engines not to crawl the pages that have the /search/label/ in their URL. This ensures that only the category pages are indexed and not any other search pages.

You can copy this code and paste it into the "Custom robots.txt" section under "Search preferences" in your Blogger dashboard. Remember to click "Save Changes" once you've made the changes.

Once you've updated your robots.txt file, it may take a few days for search engines to crawl and index your category pages. I hope this helps! If you have any further questions, feel free to ask.

How to index all Posts and Categories on blogger website

        If you want to allow search engines to crawl and index not only the categories, but also all the articles on your Blogger website, you can use the following code in your robots.txt file:

User-agent: *

Disallow:

Sitemap: http://yourblog.blogspot.com/sitemap.xml


The Disallow: line with a blank value allows search engines to crawl and index all the pages on your website. The Sitemap line tells search engines the location of your sitemap file. The sitemap is a file that lists all the pages on your website that you want search engines to crawl and index.

Follow The steps below to Add Sitemap to your blogger website

  • Go to the "Settings" section of your Blogger dashboard.
  • Click on "Search preferences."
  • Under the "Crawlers and indexing" section, click on "Edit" next to "Custom robots.txt."
  • In the "Custom robots.txt" section, add the above code.
  • Go back to "Search preferences" and click on "Edit" next to "Custom robots header tags."
  • Add the following code to the "Description" field:

<link rel="alternate" type="application/rss+xml" title="Your Blog's Title" href="http://yourblog.blogspot.com/feeds/posts/default?alt=rss" />

<link rel="alternate" type="application/atom+xml" title="Your Blog's Title" href="http://yourblog.blogspot.com/feeds/posts/default" />

This above code will add RSS and Atom feed links to your website, which will help search engines discover your content.

  • Click "Save Changes" to update your website's settings.
  • Once you've made these changes, it may take some time for search engines to crawl and index your website. Be patient and monitor your website's search engine rankings over time.

How To Fix Robot.txt errors in blogger website

            If you have updated your robots.txt file and it's still not indexing, there could be a few reasons why this is happening. Here are some things you can check:

5ways to Fix Robot.txt errors in blogger

  1. Check your robots.txt file
  2. Check your website's meta tags
  3. Check for any crawl errors
  4. Check your website's content
  5. Check your website's backlinks

Check your robots.txt file: 

Make sure that your robots.txt file is correctly formatted and doesn't contain any syntax errors. You can use the Google Search Console to check your robots.txt file and see if there are any errors.

Check your website's meta tags: 

Make sure that your website's meta tags are correctly set up. You can use a tool like the Meta Tag Analyzer to check your website's meta tags and see if there are any issues.


Check for any crawl errors:

 Use Google Search Console to check for any crawl errors on your website. If there are any crawl errors, fix them and resubmit your sitemap to Google.


Check your website's content:

 Make sure that your website's content is high-quality and unique. If your content is not unique, search engines may not index your website.


Check your website's backlinks:

 Make sure that your website has high-quality backlinks. High-quality backlinks will help search engines discover and index your website.

            If you have checked all of the above and are still having issues with indexing, you may want to reach out to a professional SEO expert or the Google Search Console support team for further assistance.


Post a Comment

Previous Post Next Post