Index Coverage: Understanding Why Google Indexes (or Doesn't Index) Your Pages
Index coverage is the question of which of your pages Google has in its index and why the others are missing. Search Console answers it in the report that used to be called Index Coverage and is now the Page indexing report. The report counts the URLs Google knows about as indexed or not indexed, gives a reason for each group that is not, and lists example URLs for each reason, limited to 1,000 per list and not guaranteed to include every URL. Reading those reasons correctly tells you which pages need work, which are excluded on purpose, and which are not worth indexing at all.
Indexed, Excluded on Purpose, Excluded by a Problem
Google’s help page describes the report’s job as showing “which pages Google can find and index on your site.” In practice its reasons fall into three groups:
- Indexed: the page is in Google’s index and can appear in results. Being indexed does not mean ranking for any particular query.
- Excluded on purpose: a
noindexrule, a canonical pointing elsewhere, or a redirect kept the URL out, as intended. A robots.txt block also shows up in the report, but it stops crawling, not indexing (see below). - Excluded by a problem: a server error, a broken redirect, an access block, or a page Google read and decided not to index.
A large “not indexed” total is not bad news by itself. Much of it can be duplicates, filtered or sorted variants, and pages nobody meant to rank.
The Statuses and What They Mean
| Status | What Google's help page says | What to do |
|---|---|---|
| <!–INLINECODE1–> | Found but not crawled yet; Google "wanted to crawl the URL but this was expected to overload the site" | Check server capacity and response times; remove crawl waste elsewhere on the site |
| <!–INLINECODE2–> | Crawled but not indexed; it "may or may not be indexed in the future" | Start with the content (thin, duplicative, not distinct), then check canonical, rendering, and internal links |
| URL marked 'noindex' | A <!–INLINECODE3–> rule kept it out | Confirm it was intended |
| URL blocked by robots.txt | Blocked by the site's robots.txt file | Confirm it was intended |
| Alternate page with proper canonical tag | Marked as an alternate of another page | Nothing, if the page it points to is the canonical you intended |
| Duplicate without user-selected canonical | A duplicate with no preferred canonical declared | Declare a canonical |
| Duplicate, Google chose different canonical than user | You marked it canonical, but Google thinks another URL is better | Align internal links, sitemaps, and redirects with your choice |
| Page with redirect | A non-canonical URL that redirects elsewhere | Nothing if the redirect is intended |
| Soft 404 | Google thinks the page is a soft 404 | Return a real 404 for missing pages, or add real content |
| Not found (404) | The page returned a 404 | Fine if intended; redirect or restore if not |
| Server error (5xx) | The server returned a 500-level error | Fix the server or hosting problem |
| Redirect error | Chain too long, loop, URL too long, or a bad URL in the chain | Point redirects straight to the final URL |
| Blocked due to access forbidden (403) or unauthorized request (401) | Googlebot was refused access | Remove access barriers from public pages |
| URL blocked due to other 4xx issue | Another 4xx error | Check the actual status code |
| Page indexed without content | Indexed, but Google could not read the content | Test rendering with URL Inspection |
A large count is not the same as a large problem. Forty thousand URLs under “Alternate page with proper canonical tag” need nothing if those canonicals are the intended ones; four hundred under “Server error (5xx)” need attention now.
Discovered Versus Crawled
These two “not indexed” statuses have different causes and different fixes.
Discovered - currently not indexed means Google knows the URL but has not fetched it. Google’s help page explains: “Typically, Google wanted to crawl the URL but this was expected to overload the site; therefore Google rescheduled the crawl. This is why the last crawl date is empty on the report.” Nothing about the content has been judged yet. Start with crawl capacity and demand: is the server fast and stable, is crawling being spent on endless filter or parameter URLs, and are the pages you care about linked from pages Google already crawls and listed in an accurate sitemap?
Crawled - currently not indexed means Google fetched and read the page and did not add it. The help page adds that it “may or may not be indexed in the future; no need to resubmit this URL for crawling.” Start with the content, without ruling out other causes: compare the page with similar pages on the site that are indexed, check in URL Inspection which canonical Google chose and whether the rendered page shows the content, and check how well the page is linked from the rest of the site. Thin text, heavy boilerplate, near-duplicate templates such as location or tag pages, and content that only appears after JavaScript runs are the places to look. Sometimes the right answer is that the page does not need to be in the index.
Each status narrows where to look first; it does not name the cause. Confirm with URL Inspection before rewriting a page Google has not read, and for a page Google read and declined, check content, canonical, rendering, and internal links together before choosing a fix.
Exclusions That Are Working as Intended
noindex. Google’s guidance on blocking indexing explains that the page must stay crawlable for the rule to be seen; a page blocked in robots.txt can keep itsnoindexhidden from Google.- robots.txt. Blocking crawling keeps Google from reading a page, but a blocked URL can still be indexed from links elsewhere, because robots.txt controls crawling, not indexing.
- Canonicalization. When several URLs serve the same content, Google chooses one canonical and lists the others as duplicates or alternates.
- Redirects. The redirecting URL shows as “Page with redirect”; its destination is what gets indexed.
Fixing Technical Blockers
Server errors, redirect errors, soft 404s, and 401 or 403 responses are not judgments about content. They are failures to fix at the server or configuration level, which can make them the quickest items in the report to clear.
Validating Fixes and Requesting Recrawls
After fixing an issue, use the report’s validation option to ask Google to recheck the affected URLs, and watch the validation status it shows. For an individual important page, the URL Inspection tool can request indexing. Google’s guide to asking for a recrawl sets expectations: “Crawling can take anywhere from a few days to a few weeks.” It adds: “Requesting a crawl does not guarantee that inclusion in search results will happen instantly or even at all.” There is also a quota for individual URL requests, so repeated requests for the same URL do not speed it up.
Sitemaps and the Report
Filter the report by sitemap to see whether the URLs you submitted are the ones being indexed. A sitemap is a hint, not an instruction; Google’s sitemap documentation states that submitting one does not guarantee that Google will download it or use it to crawl the site’s URLs. Keep it to canonical, indexable URLs, and remove URLs you have since redirected, blocked, or marked noindex.
Large Sites
On sites with very many URLs, faceted navigation and fast publishing can produce more URLs than Google will crawl. Block or consolidate low-value filter combinations, keep new important pages linked from frequently crawled ones, and watch whether the Discovered - currently not indexed count grows faster than it clears.
A Review Routine
- Check the report on a schedule that matches how often the site changes.
- Watch trends, not single snapshots.
- Identify the exact status and URLs before changing anything.
- Validate after fixing and allow time for recrawling.
- When the same exclusion keeps returning, fix the process that creates it, such as a template that re-adds
noindex.
Questions About Index Coverage
Does a high “not indexed” count hurt the pages that are indexed?
Not directly. Indexed pages are assessed on their own. Large numbers of duplicate or low-value URLs can still take up crawling that would otherwise reach the pages you care about.
What share of my URLs should be indexed?
Google publishes no target. The useful question is whether the pages built to answer real searches are indexed, after setting aside duplicates, variants, and pages excluded on purpose.