I had a problem with one of my experimental black hat blogs.
It was over three weeks old and I had submitted the sitemap to Google using Webmaster Tools, but no pages were appearing in the Google index when I did the site:yourdomainname.com.au command.
Something was wrong.
I then stumbled over a useful SEO Crawler tool at https://robhammond.co/tools/seo-crawler
What is does is check the pages on your site up to 350 maximum – probably enough for most people to manage.
There’s several page metrics it checks, including:
- Page title
- HTTP status code
- Details of redirect chains
- Meta robots details
- Followed external links & total links on-page
- rel=canonical URLs
- Any automatically detected SEO issues
You can also save the data down to a spreadsheet file for further analysis.
Ah Ha ! … instantly I saw the problem, all the pages were marked noindex,follow which tells Google to follow the links on the pages, but do not put the pages in the search results.
I then logged into the black hat blog and checked the reading settings in the WordPress Dashboard.
Sure enough this WordPress site was set to: Discourage search engines from indexing this site
I then deselected that option, saved the settings and headed back to the SEO Crawler to test again. All fixed.
Within a week all the pages were now in the Google index.
Head on over to https://robhammond.co/tools/seo-crawler and give it a try..
