Before we fix this lets understand difference between Indexed, though blocked by robots.txt and Page indexing Blocked by robots.txt
“Indexed, though blocked by robots.txt” and “Page indexing blocked by robots.txt” refer to different states of a web page’s indexing status with respect to a search engine’s web crawler.
“Indexed, though blocked by robots.txt” means that the web page is indexed by the search engine, but its content is restricted from being crawled by the web crawler. This occurs when the web page is listed in the “robots.txt” file with a “Disallow” directive, but the search engine has already crawled and indexed the page before the directive was put in place.
On the other hand, “Page indexing blocked by robots.txt” means that the web page is not indexed by the search engine because the web crawler is unable to access it. This occurs when the web page is listed in the “robots.txt” file with a “Disallow” directive and the web crawler has not yet crawled the page. As a result, the page will not appear in search engine results, affecting its visibility and ranking.
In conclusion, “Indexed, though blocked by robots.txt” means that the page is indexed but not accessible to the web crawler, while “Page indexing blocked by robots.txt” means that the page is not indexed and not accessible to the web crawler.
You can read about page indexing blocked by robots.txt
To fix “Indexed, though blocked by robots.txt”, you need to update the “robots.txt” file on your website to allow the search engine’s web crawler to access the page. Here’s how:
- Locate the “robots.txt” file: This file is usually located in the root directory of your website, e.g., “www.gomahamaya.com/robots.txt“. You can access it by entering the URL in your browser.
- Edit the “robots.txt” file: Open the file in a text editor and remove the “Disallow” directive for the specific page that’s causing the indexing error. For example, if the block looks like “Disallow: /example-page/”, simply delete the line.
- Save and upload the changes: Save the changes you made to the “robots.txt” file and upload it to your server.
- Submit a reindex request: After making changes to the “robots.txt” file, you may need to submit a request to the search engine to reindex your website. This will allow the search engine to discover the changes you made and update its index accordingly.
Please note that changes to the “robots.txt” file may take some time to take effect, as it may take a while for search engines to recrawl your site.
https://www.youtube.com/watch?v=KhSxE1xO2q8