What Is Duplicate Content? Understanding Content Uniqueness Issues
Duplicate content is substantially the same content available at more than one URL, on one site or across several. Duplication on its own is not a penalty matter: when Google finds pages with the same content, it groups them and chooses one to show. The cost of unmanaged duplication is that Google, not you, picks the version that appears, signals split across copies, and crawling goes to repeated pages. Understanding uniqueness issues means separating the causes, because each has a different fix.
How Google Handles Duplicates
Google’s overview of how Search works describes the step: to pick a canonical, Google first groups the pages it found with similar content, a process it calls clustering, and then selects the one most representative of the group. The other pages in the cluster are alternates that “may be served in different contexts,” such as on mobile.
When one URL becomes canonical, signals such as links to the others can be consolidated with it. Google’s documentation on consolidating duplicate URLs explains that it helps search engines “consolidate the signals they have for the individual URLs (such as links to them) into a single, preferred URL.” Without clear signals from you, Google may choose a URL you would not have chosen.
Duplication becomes a spam matter when it is used to manipulate rankings. Google’s spam policies describe scraping as “the practice of taking content from other sites, often through automated means, and hosting it with the purpose of manipulating search rankings,” and doorway abuse as “when sites or pages are created to rank for specific, similar search queries.”
No Similarity Percentage
Google publishes no percentage at which pages count as duplicates. Asked in September 2022 whether such a number exists, John Mueller of Google answered, as Search Engine Journal reported: “There is no number (also how do you measure it anyway?).”
Technical Causes
| Cause | Example | Fix |
|---|---|---|
| Tracking parameters | <!–INLINECODE0–> and <!–INLINECODE1–> | Canonical to the clean URL; link internally without parameters |
| HTTP and HTTPS | <!–INLINECODE2–> and <!–INLINECODE3–> | Redirect HTTP to HTTPS |
| www and non-www | <!–INLINECODE4–> and <!–INLINECODE5–> | Redirect to one host |
| Trailing slash | <!–INLINECODE6–> and <!–INLINECODE7–> | Redirect to one form |
| Session IDs | <!–INLINECODE8–> | Keep session state in cookies |
| Print versions | <!–INLINECODE9–> and <!–INLINECODE10–> | Canonical from the print version to the article |
| Indexed staging sites | <!–INLINECODE11–> | Password protection on non-production environments |
Search Console no longer has a setting for parameters. Google announced on March 28, 2022, that the URL Parameters tool would be removed from Search Console a month later. The controls that remain are canonical tags, consistent internal links to the canonical URL, redirects, and robots.txt rules for parameter patterns that produce useless URLs.
Paginated pages are not duplicates of page one; Google’s pagination guidance asks for each page to have its own canonical URL.
Editorial Causes
- Syndication. Republishing an article on a partner site is legitimate, but Google’s canonicalization troubleshooting guidance no longer recommends a canonical tag for it: “The canonical link element is not recommended for those who want to avoid duplication by syndication partners, because the pages are often very different. The most effective solution is for partners to block indexing of your content.”
- Scraping. Other sites copying your content without permission. Copyright holders can send takedown notices under the DMCA to the host or to Google.
- Boilerplate. Disclaimers, shipping terms, and about text repeated across pages are normal; the issue arises when they outweigh the unique content on a page.
- Manufacturer descriptions. Retailers using the same supplied product text share that block with every other retailer. Price, stock, delivery terms, reviews, and their own specifications still make the pages differ, but the shared text adds nothing of the retailer’s own.
- Thin location variants. City pages that swap a place name into the same template add nothing for each location, and pages created to rank for similar searches can fall under the doorway policy.
E-Commerce Patterns
Online stores generate duplicates through product variants with their own URLs and the same description, filter combinations in faceted navigation, and the same listing on marketplaces. Choose one canonical URL for each distinct product or category view, point variants to it, and save separate indexable pages for views that serve a different need, such as a distinct size range or audience.
Language Versions Are Not Duplicates
Translated versions of a page serve different audiences. Connect them with hreflang and let each version be canonical to itself; collapsing them with canonical tags would remove the pages from the results their audiences see. Same-language regional copies are a different case. Google’s canonicalization overview lists region variants with essentially the same content in the same language, such as separate US and UK pages, among duplicates, and its multi-regional guidance says to pick a preferred version, using rel="canonical" together with hreflang. Regional pages that differ in substance, such as prices, currency, or delivery terms, can stand as separate canonical pages.
Finding Duplicates
- Check the Page indexing report for “Duplicate without user-selected canonical” and “Duplicate, Google chose different canonical than user.”
- Crawl the site to find identical titles, identical body text, and parameter URLs.
- Use URL Inspection on key pages to see which canonical Google selected.
- Fix the cause at its source, then point internal links and the sitemap at the canonical URLs.