Apr 07, 2009 04:45 AM - John P
I have just checked my backlinks using SmartPageRank. When I clicked on the "Show Nofollow" option, all of the Prospotlight links came up as Nofollow with the explanation "Forbidden by Robots.txt".
Is this a deliberate policy or is it a mstake or is the software tool mistaken? Sorry to be a nuisance, but I would be grateful if someone could clarify as I had thought that Prospotlight links were dofollow.
Many thanks.
Tags: Nofollow
Apr 07, 2009 12:40 PM - Brian Broderick said:
Thanks for noticing this. We had a default robots.txt file that had everything commented out. Your software might not have caught the comment character. Even still, I've replaced that with the standard do-follow text. Please try your software again and let me know if it's still showing nofollow.
Thanks,
Brian
Apr 08, 2009 04:26 PM - John P said:
Many thanks for checking this out so quickly. I'd hoped it was something simple like that.
I have just checked the Smartpagerank tool again and one page is still showing all 4 links as nofollow; the other page is showing 2 links nofollow and 2 links dofollow. Mind you, it will probably take time for the search engines to recrawl the pages.
Thanks again.
John
Apr 09, 2009 11:30 AM - Brian Broderick said:
Just to be sure, I checked Google's webmaster tools to see if they aren't crawling due to the robots.txt file. Here is their message:
"We have no errors to report. We crawl regularly, so check back later to see updates."
It showed 0 URLs blocked due to a robots.txt file.