Index Coverage: Understanding Why Google Indexes (or Doesn’t Index) Your Pages

Executive Summary

A page existing on your server has nothing to do with whether Google has it in its index. Those are two separate questions, and conflating them is the single most common reason site owners panic over numbers that aren’t actually a problem. The tool for answering the real question, “is this page indexed, and if not, why not,” is Google Search Console’s Page indexing report (the report Google used to call the Index Coverage report; the name changed, the underlying data did not). It breaks every URL Google knows about into one of a fixed set of statuses, and those statuses fall into three broad buckets: indexed, excluded on purpose, and excluded because something is wrong.

The bucket that causes the most confusion is the middle one labeled “not indexed,” because it actually contains two statuses that get treated as interchangeable when they describe different problems with different fixes. “Discovered – currently not indexed” means Google found the URL but has not crawled it yet. “Crawled – currently not indexed” means Google did crawl it, looked at the content, and chose not to index it anyway. The first is a queue problem. The second is closer to a quality verdict. Treating them as the same status leads to wasted effort: improving content quality won’t help a page Google hasn’t even fetched yet, and improving internal links won’t fix a page Google already evaluated and rejected.

A few things worth keeping in mind before digging into specific statuses:

• A growing “not indexed” count is not automatically bad. Many of those URLs are duplicates, filtered/sorted variants, or pages you never intended to rank.
• The fix for “Discovered” and the fix for “Crawled” are different. Confirm which status you’re actually looking at before changing anything.
• Sitemaps are a hint, not an instruction. Submitting a URL does not obligate Google to crawl or index it.
• Requesting indexing through the URL Inspection tool is for individual important pages, not a substitute for fixing the underlying reason pages aren’t getting indexed at scale.
• Index coverage should be reviewed on a cadence that matches how often the site publishes or restructures content, not treated as a one-time check.

Page Indexing Report Fundamentals

The Page indexing report in Search Console shows, in Google’s own words, “which pages Google can find and index on your site, and learn about any indexing problems encountered.”

The report’s top-level view is a count of indexed pages plotted over time, with a breakdown beneath it of every reason a URL is not indexed. Each reason is its own status with its own count, and clicking into a status shows the actual list of affected URLs (sampled, for large sites, rather than exhaustive). You can filter the report by sitemap, which is useful for checking whether the URLs you explicitly submitted are the ones actually getting indexed, as opposed to the URLs Google found on its own through crawling and internal links.

It helps to think of every URL Google has encountered as falling into one of three buckets:

1. Indexed. The page is in Google’s index and eligible to appear in search results. Eligible does not mean guaranteed to rank or even to show up for any given query, just that it’s part of the pool Google can draw from.
2. Excluded by design. The URL was deliberately kept out of the index, through a noindex directive, a robots.txt block, or a canonical tag pointing somewhere else. These are not errors. They’re the expected outcome of decisions you (or your CMS defaults) already made.
3. Excluded by problem. Something prevented indexing that you probably didn’t intend: a server error, a broken redirect, or Google deciding not to index a page you wanted indexed. This is the bucket that deserves investigation.

The Page Indexing Statuses, Decoded

Below are the statuses you’ll actually encounter in the report, with Google’s own definitions condensed and what they typically mean for action.

Status	Bucket	What it means	Typical response
Indexed	Indexed	The page is in Google's index.	Monitor; no action needed.
Indexed, though blocked by robots.txt	Indexed (warning)	Google indexed the page anyway, usually from external links, despite a robots.txt block preventing it from reading the content.	Decide whether to actually block it (and accept a thin listing) or unblock it so Google can crawl and properly evaluate it.
Page indexed without content	Indexed (warning)	The page is indexed but Google couldn't read the content, often a rendering or access issue.	Check that the page renders correctly for Googlebot; test with the URL Inspection tool.
Discovered – currently not indexed	Not indexed	Google found the URL but has not crawled it yet.	Usually a crawl-priority issue; see the dedicated section below.
Crawled – currently not indexed	Not indexed	Google crawled the page and chose not to index it.	Usually a content-quality issue; see the dedicated section below.
URL marked 'noindex'	Excluded (intentional)	A <!–INLINECODE1–> directive told Google not to index the page.	Confirm this was deliberate; remove the tag if not.
URL blocked by robots.txt	Excluded (intentional)	robots.txt prevented crawling.	Confirm this was deliberate; update robots.txt if not.
Alternate page with proper canonical tag	Excluded (intentional)	The page correctly points to a different canonical URL.	No action; this is canonicalization working as intended.
Duplicate without user-selected canonical	Excluded	The page duplicates another page, but no canonical was specified.	Add explicit canonical tags so Google doesn't have to guess.
Duplicate, Google chose different canonical than user	Excluded	You specified a canonical, but Google picked a different URL as canonical anyway.	Investigate whether your specified canonical is actually the strongest version; align signals (internal links, sitemap) toward it.
Page with redirect	Excluded (intentional)	The URL redirects elsewhere and isn't indexed itself.	No action if the redirect is intentional.
Soft 404	Error	The page returns a 200 status but reads as an empty or "not found" page to Google.	Return a real 404/410 for genuinely removed content, or add substantive content if the page should exist.
Not found (404)	Error	The page returned a 404.	Remove from sitemap if intentional; restore or redirect if not.
Server error (5xx)	Error	The server returned a 500-level error when Google requested the page.	Fix the underlying server issue; check hosting capacity and error logs.
Redirect error	Error	A redirect chain was too long, looped, or led to an invalid URL.	Simplify the redirect chain to a single hop.
Blocked due to access forbidden (403) / unauthorized request (401)	Error	Googlebot was denied access.	Remove unintended authentication barriers on pages meant to be public.
URL blocked due to other 4xx issue	Error	The page returned a 4xx status code other than 404, 401, or 403 (a malformed request, for example).	Check the actual response code and fix the underlying server or routing issue.

That table is worth bookmarking, because the report’s default sort (by URL count) tends to put the scariest-looking number first regardless of whether it’s actually a problem. A site with 40,000 URLs in “Alternate page with proper canonical tag” has nothing to fix. A site with 400 URLs in “Server error (5xx)” has an urgent one.

Discovered vs. Crawled: Two Different “Not Indexed” Problems

This distinction gets blurred constantly, including in plenty of SEO writing that treats “not indexed” as a single undifferentiated state, so it’s worth being precise about it.

“Discovered – currently not indexed” means Google is aware the URL exists, through a sitemap, an internal link, an external link, or some other discovery path, but has not yet sent Googlebot to fetch it. Nothing about content quality has been evaluated, because nothing has been read yet. This status is fundamentally about crawl prioritization: Google’s own documentation on managing crawl budget explains that crawling decisions factor in “popularity, overall user value, content uniqueness, and serving capacity,” and that the crawl capacity Google extends to a site fluctuates with how quickly and reliably that site responds. A new page on a small, infrequently-crawled site, or a page buried deep in a large site’s structure with few internal links pointing to it, can sit in “Discovered” for a while simply because Google hasn’t gotten around to it, not because Google has judged it unworthy.

“Crawled – currently not indexed” is a different situation entirely. Google has fetched the page, read the content, and made an active decision not to add it to the index. That’s a content and quality signal, not a queue position. A page can move from “Discovered” to “Crawled – currently not indexed” without ever becoming “Indexed,” which is the scenario worth paying attention to: it means the crawl happened, Google had the chance to index the page, and declined.

The practical difference matters because the fixes don’t overlap much:

• If a page is stuck in “Discovered,” the lever to pull is usually about getting Googlebot to prioritize the crawl: stronger internal linking from already-indexed pages, sitemap inclusion, and overall site crawl health (fast server response, no crawl budget being wasted on low-value duplicate URLs elsewhere on the site).
• If a page is stuck in “Crawled – currently not indexed,” the lever to pull is the content itself: is it thin, near-duplicate of something else on the site, or simply not differentiated enough to clear whatever bar Google is applying to similar pages on similar sites.

Conflating the two leads to the wrong diagnosis in both directions: spending a content-rewrite cycle on a page Google has never actually read, or spending an internal-linking cycle on a page Google already read and rejected.

Common Exclusion Reasons That Aren’t Problems

Not everything in the “not indexed” totals is bad news. A meaningful share of most sites’ excluded-URL counts is working exactly as designed.

noindex directives. If a page carries a noindex meta tag or X-Robots-Tag header, Google will, per its own documentation, “drop that page entirely from Google search results, regardless of whether other sites link to it,” and one requirement worth remembering is that the page must still be crawlable for that directive to work; a page blocked by robots.txt can hide its own noindex tag from Google, which produces the confusing “indexed despite being blocked” warning status. Internal search results, filtered/sorted product listings, thank-you pages, and staging-style utility pages are common, legitimate uses of noindex.

robots.txt blocks. If crawling itself is blocked, the URL won’t be indexed through normal crawling (though, as covered above, it can still appear in results via external links if Google never sees the block reversed). This is appropriate for admin paths, internal search, and similar low-value crawl targets.

Canonicalization. When multiple URLs serve essentially the same content (a common situation with parameter-based sorting, filtering, or tracking parameters), Google consolidates them and chooses a canonical version using signals including redirects, rel="canonical" annotations, sitemap inclusion, and an HTTPS preference when other signals are equal. The non-canonical duplicates show up as excluded, which is the system functioning correctly, not a sign something is broken. The one variant worth checking is “Duplicate, Google chose different canonical than user,” which means Google overrode your explicit preference; that’s worth investigating, since it usually means your specified canonical isn’t backed by strong enough signals (internal links and sitemap entries still pointing at the non-canonical version is a frequent cause).

Redirects. A URL that 301-redirects elsewhere correctly shows as “Page with redirect” rather than indexed; the redirect target is what gets indexed, not the redirecting URL.

Diagnosing “Discovered – Currently Not Indexed”

Since this status is fundamentally a prioritization problem, the diagnostic questions are about how much crawl priority the page is realistically earning, not about the page’s content in isolation.

• How is the page linked internally? A page with no internal links pointing to it, reachable only through the sitemap, sends a weak signal of importance. Pages linked from high-traffic, frequently-crawled pages tend to get prioritized faster.
• Is the broader site competing for crawl attention? A large site generating enormous numbers of low-value URLs (faceted navigation combinations, session parameters, infinite pagination) spreads Google’s attention thin, which can delay crawling of legitimate new content elsewhere on the same site.
• Is the server fast and reliable? Slow response times and intermittent errors reduce the crawl capacity Google is willing to extend, independent of how good the content is.
• Is the page actually in the sitemap, and is the sitemap current? A sitemap is a hint, not a guarantee of crawling, but an accurate, current sitemap remains one of the more reliable discovery signals available.

If those factors look reasonable and the page still sits in “Discovered” for an extended stretch, the realistic options are improving the signals (links, sitemap accuracy, site-wide crawl health) or accepting that this particular page isn’t a crawl priority for Google given everything else competing for the same budget.

Diagnosing “Crawled – Currently Not Indexed”

This is the harder status to fix, because it means Google already made a quality judgment, and that judgment is rarely explained beyond the status label itself.

• Compare against pages that are indexed. What’s different about this page relative to similar pages on the same site that did get indexed? Thinner content, more boilerplate relative to unique text, and closer similarity to other pages are common patterns.
• Check for near-duplication you didn’t intend. Templated pages (location pages, tag archives, lightly-varied product variants) are frequent occupants of this status, because the unique content per page is too thin to clear the bar relative to how similar they are to each other.
• Confirm the page actually renders the content you think it does. If the content depends on client-side JavaScript that doesn’t render reliably, Google may be evaluating a page that looks far thinner than what a human visitor sees.
• Consider site-wide context. Google’s quality assessment isn’t purely page-by-page; a site with a lot of thin or low-value content can see that context affect borderline pages elsewhere on the same domain.

The honest resolution in some cases is that the page genuinely doesn’t warrant indexing on its own. Not every URL a CMS generates needs to compete for a place in Google’s index, and treating every “Crawled – currently not indexed” page as a bug to be fixed, rather than sometimes a correct call, leads to chasing indexation for pages that wouldn’t perform even if indexed.

Technical Blockers: Errors, Redirects, and Soft 404s

A separate category of “not indexed” has nothing to do with content quality or crawl prioritization and everything to do with the page being technically broken from Google’s perspective.

• Server errors (5xx). If Googlebot consistently gets a server error, indexing is off the table regardless of content. Persistent server errors during crawl attempts can also reduce Google’s overall crawl capacity allocation for the site, compounding the problem beyond the individual affected URLs.
• Redirect errors. Chains that loop, run too long, or terminate somewhere invalid prevent Google from resolving the redirect to a final, indexable destination. A single clean hop is far more reliable than a chain accumulated through successive site migrations.
• Soft 404s. A page that returns an HTTP 200 but reads, to Google, like an empty or “not found” page (a common pattern on out-of-stock product pages or sparsely-populated category pages) gets flagged as a soft 404 rather than indexed. The fix is either a genuine 404/410 status for content that’s actually gone, or enough real content to stop reading as empty.
• Access errors (401/403). Authentication or permission barriers that block Googlebot specifically (as opposed to intentionally gating content behind a real login) prevent crawling entirely.

None of these are quality judgments. They’re technical failures, and they’re usually the fastest category to fix once identified, since the remedy is typically a server-side or configuration change rather than a content rewrite.

Validating Fixes and Requesting Indexing

Once an issue is fixed, the natural next step is telling Google so it doesn’t have to wait for the next scheduled crawl to notice. The Page indexing report includes a “Validate Fix” option on each status that triggers Google to recheck the affected URLs and report back on whether the issue is resolved.

For individual important pages, the URL Inspection tool lets you request indexing directly. Google is explicit about the limits of this: “Submitting a request does not guarantee that the page will appear in the Google index,” and the tool itself notes that indexing typically happens within roughly a day in many cases, but can take considerably longer, with no fixed guarantee either way. It’s also rate-limited and intended for individual high-value URLs (a newly published flagship page, or a page that just had a significant fix applied), not as a substitute for fixing whatever systemic issue is keeping a larger set of pages out of the index.

Sitemaps and Page Indexing Alignment

A sitemap’s job is communicating which URLs you consider worth indexing, but it’s worth being precise about what that actually accomplishes. Google’s own sitemap documentation states plainly that submitting a sitemap “doesn’t guarantee that Google will download the sitemap or use the sitemap for crawling URLs on the site.” It’s a signal among several, not an instruction.

That said, comparing your sitemap against the Page indexing report’s coverage is one of the more useful diagnostic habits available. If a meaningful share of submitted sitemap URLs show up as “Discovered – currently not indexed” or “Crawled – currently not indexed” rather than “Indexed,” that’s a concrete, specific list of underperforming pages rather than an abstract concern. And the inverse check matters too: a sitemap that includes URLs you’ve since blocked, redirected, or marked noindex sends a contradictory signal and is worth periodically cleaning up.

Page Indexing at Scale: Large Sites and Faceted Navigation

Everything above applies at any site size, but the dynamics shift once a site has hundreds of thousands or millions of URLs, because crawl budget stops being a theoretical concern and starts directly determining how much of the site Google can realistically keep up with.

Faceted navigation (filtering and sorting by color, size, price range, and similar attributes) is the most common source of runaway URL counts on large e-commerce and listing sites, because each combination of filters can generate a technically distinct URL. Left unmanaged, this multiplies the number of URLs Google has to consider far beyond the number of pages actually worth indexing, which dilutes crawl attention away from the pages that matter. The standard responses are blocking low-value parameter combinations, consolidating near-duplicate facet pages with canonical tags, and being deliberate about which facet combinations are actually worth letting Google discover at all.

Content velocity matters too. A site adding thousands of new pages weekly needs internal linking and sitemap structure to actively surface the newest, most important additions, rather than relying on Google to eventually stumble onto them through general crawling. At scale, “Discovered – currently not indexed” backlogs tend to grow faster than they clear unless the site actively manages what’s competing for the same crawl budget.

A Practical Monitoring Workflow

• Check the Page indexing report on a cadence that matches your publishing rhythm. A site adding new content daily benefits from more frequent review than one that publishes occasionally.
• Watch trends, not just snapshot counts. A status count that’s stable or shrinking is a different situation than one growing month over month, even at the same absolute number.
• Diagnose before fixing. Confirm which specific status and which specific URLs are affected before changing robots.txt, canonical tags, or content, since the wrong fix for the wrong status wastes effort and can introduce new problems.
• Use the sitemap-filtered view to check whether the URLs you explicitly care about are the ones actually getting indexed.
• Validate after fixing, using the report’s Validate Fix feature, and give Google time to actually recrawl before assuming a fix didn’t work.
• Document recurring issues. If the same exclusion reason keeps reappearing after fixes (a CMS that re-adds noindex tags on republish, for instance), the fix belongs upstream in the workflow that’s causing it, not in repeated manual corrections.

Frequently Asked Questions

Why aren’t my new pages showing up as indexed yet?
New pages have to be discovered, crawled, and then judged worth indexing, and any of those three steps can be the holdup. Check the Page indexing report for the specific status first; “Discovered – currently not indexed” points to a crawl-priority issue, while “Crawled – currently not indexed” points to a content issue. Strengthening internal links and confirming sitemap inclusion address the first; improving content depth and differentiation addresses the second.

How long should I wait before treating a “not indexed” page as a real problem?
Google doesn’t publish a fixed timeline, and it varies considerably by site authority, crawl frequency, and how competitive the site’s overall crawl demand is. If weeks have passed, the page is correctly included in an accurate sitemap, internal links point to it, and there’s no technical block, it’s reasonable to start investigating rather than continuing to wait indefinitely.

Should I request indexing for every page that isn’t indexed yet?
No. The URL Inspection tool’s request-indexing feature is built for individual, important URLs, not for clearing a backlog of dozens or hundreds of pages. Fixing the underlying crawl or quality issue is the actual solution for anything beyond a handful of pages.

What’s the difference between “not indexed” and a page that was indexed and later removed?
“Not indexed” statuses (Discovered, Crawled, blocked, etc.) describe URLs that have never been added to the index. A page that was indexed and later dropped, sometimes called deindexing, is a separate situation, typically caused by a noindex tag added after the fact, a manual action, a quality reassessment, or the page becoming inaccessible. The Page indexing report’s history view can show whether a URL was ever indexed in the past, which helps tell the two situations apart.

Is “Discovered – currently not indexed” always a crawl-budget issue?
Mostly, but not exclusively. Crawl prioritization (driven by site responsiveness, perceived popularity, and how much of the site’s overall crawl demand is being spent on duplicate or low-value URLs) is the primary mechanism Google describes for this status. A page can also sit here longer than expected simply because it’s new and hasn’t accumulated enough internal or external signals yet to register as a priority.

Can robots.txt cause pages to appear in “Discovered – currently not indexed”?
No, that combination doesn’t happen. If robots.txt blocks a URL, it shows up under “URL blocked by robots.txt,” a distinct status, not “Discovered.” “Discovered” specifically means Google could attempt the crawl and simply hasn’t yet, not that crawling is being prevented.

Does a high “not indexed” count hurt rankings for the pages that are indexed?
Not directly. The pages that are indexed are evaluated and ranked on their own merits. Where it can matter indirectly is at scale: a site that’s flooding Google with large volumes of duplicate or thin URLs spends crawl budget that could otherwise go toward discovering and re-crawling the pages that actually matter, and in aggregate can contribute to the kind of site-wide quality context that affects borderline pages.

How many pages should realistically be indexed on my site?
There’s no target ratio Google publishes, and chasing a high percentage of total URLs indexed is the wrong goal in itself. The more useful question is whether the pages that should be indexed (the ones built to answer a real query or serve a real visitor) actually are, regardless of what the percentage looks like once filtered duplicates, parameter variants, and intentionally excluded utility pages are accounted for.

If you haven’t opened the Page indexing report recently, start there: sort by status count, set aside anything in an “excluded by design” bucket, and focus your first pass entirely on the error and not-indexed statuses that actually warrant a fix.