Quick Answer: How Do I Find Pages Google Isn’t Indexing?
Short answer: open Google Search Console, go to Indexing > Pages, and switch the dropdown above the chart from “All known pages” to “All submitted pages”. The “Why pages aren’t indexed” table then lists the pages you actually care about that Google has left out, along with the reason. Before changing anything, run a few of those URLs through the URL Inspection tool to confirm the cause.
You’ve published a new service page, waited a fortnight, and it still won’t show up in Google. Not even when you search for the exact title. It’s one of the questions we hear most from business owners who run their own sites, and it’s rarely a mystery once you know where to look.
This is the second instalment of our Weekly SEO Tip series. Last week covered internal linking mistakes. This week is the process we use to find pages Google isn’t indexing, pin down why, and fix the actual cause, instead of pressing “Request indexing” and hoping. Have your Search Console login handy before you start.
First, Is It a Crawling, Indexing or Ranking Problem?
These three get muddled all the time, and mixing them up sends people off fixing the wrong thing.
Picture three doors in a row. At the first, Googlebot has to actually reach the page, and a block, a server error or the lack of any link pointing there will stop it cold. Get through, and Google then weighs up whether the page earns a place in its index. Lots don’t. Only indexed pages get to compete in ranking, which is about where you appear for a given search. A page stuck on page six of the results is through both earlier doors and simply losing the race.
This tip deals with the first two. If a page is indexed but buried, you’ve got a ranking problem, and the Page indexing report won’t tell you much about it.
Pages Not Indexed by Google? The 7-Step Check We Use
Step 1: Open the Page Indexing Report
Once you’re in, glance at the property name in the top-left corner. It’s surprisingly easy to end up in an old http:// or non-www version and wonder why the numbers look odd. Then open Indexing > Pages from the sidebar. Don’t panic at the grey number. On a typical small business site the not-indexed pile is often large and mostly harmless, full of redirects, old URLs, tag archives and tracking variations that Google has quite sensibly skipped.
Step 2: Filter to the Pages You Submitted
Most people skip this step. Click the dropdown above the chart, which reads “All known pages” by default, and select “All submitted pages”. Now you’re only looking at URLs in your XML sitemap, which should be the pages you want people to find. Anything not indexed in this view deserves a closer look.
Haven’t submitted a sitemap? Add one under Indexing > Sitemaps first. Most WordPress SEO plugins create one for you automatically.
Step 3: Read the “Why Pages Aren’t Indexed” Table
Below the chart, each row is a reason, with a count of affected pages and a Source column. “Website” means something on your site is causing it, so it’s usually yours to fix. “Google systems” generally reflects a decision on Google’s side. Click any row to see example URLs. That list tops out at 1,000 rows and isn’t guaranteed to be complete, so on bigger sites treat it as a sample.
Step 4: Confirm the Cause With URL Inspection
Before touching anything, paste one affected URL into the search bar at the top of Search Console. That opens URL Inspection. The headline will read “URL is on Google” or “URL is not on Google”. Open the Page indexing section underneath and check, in this order:
- Crawl allowed? A “No” here means robots.txt is blocking Google.
- Page fetch should say Successful. Anything else points to a server or access problem.
- Indexing allowed? If this says No, look for a noindex tag or header.
- User-declared vs Google-selected canonical. When they differ, Google thinks another URL is the main version.
Then click Test live URL. The main result is basically Google’s memory of the page, which could be days or weeks old. Test live URL goes and fetches it there and then, so yesterday’s fix only shows up there. Even a spotless live test only proves the page is indexable. Whether Google actually indexes it is still Google’s call.
Step 5: Match Each Status to a Fix
These are the statuses we run into most on small business sites. Wording in your account can differ slightly from Google’s help pages, but the meaning is the same.
URL marked “noindex”
The page is telling Google to stay out. This one turns up a lot after launches. Somebody ticks WordPress’s “Discourage search engines from indexing this site” box while the site’s under construction, the site goes live, and the box stays ticked for months. Untick it, or strip the noindex tag (occasionally it’s an X-Robots-Tag header instead) from any page you want showing up. Your thank-you page is probably better hidden, so leave that one.
Blocked by robots.txt
There’s a Disallow rule in your robots.txt covering that URL. You can see the file by adding /robots.txt to the end of your domain in any browser. Find the line matching the page’s path and remove it. Here’s the counterintuitive bit. Blocking a page in robots.txt doesn’t reliably keep it out of results, so if you want something hidden, unblock it and use noindex.
Crawled – currently not indexed
Google fetched the page and decided, for now, not to keep it. There’s no technical error to fix. Usually it’s a judgement about usefulness, and the culprits tend to be thin pages, near-copies of other pages, or location pages that only swap the town name. Google says resubmitting isn’t needed. Improving the page is what helps. Add the detail a customer would genuinely need, fold weak pages into one stronger page, and link to it from related pages on your site.
Discovered – currently not indexed
Google knows the URL exists but hasn’t crawled it yet. More often than not on a small site, the page is an orphan. It’s in the sitemap, but no menu, blog post or service page links to it. Give it a couple of links from pages Google already visits, with anchor text that says what the page is about (our internal linking tips go into this). Now and then a slow or struggling server is the reason, so glance at Settings > Crawl stats as well.
Duplicate statuses
“Duplicate without user-selected canonical” and “Duplicate, Google chose different canonical than user” are close cousins. In each case Google sees the page as a copy and has indexed another URL instead. Usually harmless. A site reachable at both www and non-www, or URLs with tracking codes tacked on, will throw up duplicates constantly. Pop the URL into URL Inspection and check which address Google selected as canonical. Fine with its pick? Move on. If not, a canonical tag pointing at the right page is the starting point, but Google treats it as a hint, so your menus, internal links and sitemap should all use the same address too. Genuinely identical pages? Merge them.
Soft 404
The page returns a normal “OK” status, yet to Google it looks empty or like an error page. Empty category pages and discontinued products are the usual suspects. Either give the page real content or, if it’s gone for good, return a proper 404 or 410, or 301 redirect it to the closest relevant page.
Statuses That Are Usually Fine
“Not found (404)”, “Page with redirect” and “Alternate page with proper canonical tag” mostly need no action. Deleted pages should 404, and redirected URLs are supposed to appear as redirects. Skim the 404 list anyway. Spot an old service page in there? Give it a 301 redirect to wherever that content lives now, because any links pointing at it currently go nowhere. And none of these URLs belong in your sitemap, so prune any still listed.
Server Error (5xx) and Redirect Error
Fix these quickly. A 5xx means your server failed when Google called, so check Crawl stats for host problems and speak to your hosting provider if it keeps happening. Redirect errors tend to follow a redesign, caused by loops or long chains. Point each old URL straight at its final destination in a single hop.
Report looking like a wall of reasons, or counts running into the hundreds? Our technical SEO team digs through Search Console data like this regularly. Book a free consultation.
Step 6: Validate the Fix, Then Request Indexing Sparingly
Once every affected page for a reason is fixed, open that reason in the Page indexing report and click Validate fix. Google then works back through the affected URLs. Google’s guidance puts this at up to about two weeks. If it finds a single page that’s still broken, validation fails, which is why it pays to fix every instance before you start.
Request indexing in URL Inspection is fine for one or two important pages. It nudges Google to crawl the URL, but you only get a limited number per day, and Google says outright that it won’t guarantee the page makes it into the index. With dozens of URLs, skip it. An up-to-date sitemap covers them.
Step 7: Recheck Once a Month
Indexing issues have a habit of coming back. A routine plugin update can quietly switch noindex on again. A recurring calendar reminder does the trick: open the report, set the filter to submitted pages, jot down the not-indexed number. It’ll drift a little from month to month. A sudden jump is worth chasing the same day.
Frequently Asked Questions
Why is Google not indexing my website?
Usually it’s a noindex tag left on after launch, a robots.txt block, pages Google judged too thin or too similar to index, or pages it hasn’t crawled yet because little links to them. The Page indexing report in Search Console shows which applies to each URL.
How long does it take Google to index a new page?
There isn’t a set timeframe. Search Console’s own help suggests about a week or so just for crawling, and indexing can take longer after that.
Does “Request indexing” guarantee my page will appear in Google?
No. It asks Google to crawl the URL, within a daily quota. Google still decides whether the page is indexed.
What is the difference between “Crawled – currently not indexed” and “Discovered – currently not indexed”?
With “Crawled”, Google has already visited the page and passed on indexing it for now, so the fix is usually better content. “Discovered” sits a step earlier. Google has the URL on its list but hasn’t been to look yet, which often happens when hardly any pages link to it.
Do I want every page on my site in Google’s index?
No. Thank-you pages, login areas, internal search results and filtered or tag URLs are often better left out. What matters is that the pages you’d want a customer to land on from Google are indexed.
Before You Close Search Console
If there’s one habit to take from this, it’s checking before fixing. Filter to submitted pages and inspect the URL before you change a thing, because the cause is rarely exotic and usually sits right there in the report. Do the seven steps once. After that, the monthly check should catch most problems before they cost you traffic.
Not keen on doing it yourself? Send it our way and we’ll go through your Search Console data, work out why pages are being left out and put the fixes in place. Book a free consultation, or read more about our SEO services.
