Duplicate Content

Duplicate content isn't a penalty; it's a selection problem. When multiple URLs serve identical or near-identical content, Google has to pick one to rank. If your signals are unclear or contradictory, Google picks wrong (or picks inconsistently), your PageRank gets split between versions, and none of them rank as well as they could.

The fix isn't panic. It's clarity.

Where duplication silently comes from#

Most duplicate content isn't intentional. It's a side effect of how websites are built.

HTTP vs. HTTPS#

http://example.com/page and https://example.com/page both returning 200. Fix: server-level 301 redirect of all HTTP to HTTPS, permanently.

www vs. non-www#

www.example.com and example.com both live. Fix: pick one canonical version and 301 the other. Search Console's old preferred domain setting was retired in 2019; redirects and canonicals are how you tell Google now.

Trailing slash variants#

/page and /page/ returning identical content. Fix: pick one, 301 the other universally across the site.

URL parameters: the big one for e-commerce#

Faceted navigation and sorting generate URLs like:

/shoes?color=red&size=10
/shoes?size=10&color=red   ← same result, different order
/shoes?color=red&size=10&sort=price
/shoes?utm_source=email&color=red&size=10

A single category page with 10 filter options and 3 sort orders can silently generate hundreds of duplicate URLs. Left unmanaged, this eats crawl budget and splits authority across URLs that should all be pointing to one canonical.

Fix options:

<link rel="canonical"> on parameter URLs pointing to the clean base URL
robots.txt Disallow for parameter combinations that add no indexable value (Google's preferred fix for crawl waste from faceted navigation)
Server-side redirect of known duplicate parameter patterns (for example, normalising parameter order)

Pick one per URL pattern. A canonical on a URL that robots.txt blocks is never read.

Syndicated content#

You publish something. It gets republished on another site. Now two pages compete for the same query. Google usually picks the original, but "usually" and "always" are different things, especially if the syndicating domain has more authority than yours.

If your content goes elsewhere: since May 2023 Google's guidance has been that the most effective fix is for the partner to noindex their copy; a cross-domain canonical often fails because the syndicated page looks too different (other templates, ads, related links). Ask for noindex first, a canonical to your original as the fallback, and at minimum a clear link back. Publish on your own site first and request indexing with URL Inspection so your version is found first.

If you republish others' content: Add rel="canonical" on your copy pointing to the original, or noindex the page if it's purely for user convenience.

Thin location and service pages#

"We provide [service] in [city]" × 50 cities, all identical except the city name. Beyond duplication, this matches Google's doorway abuse spam policy. Either write genuinely unique content for each location, or consolidate to a single regional service page.

Session IDs and tracking parameters#

URLs with appended session identifiers like ?sessionid=abc123 or tracking parameters that change the URL without changing the content. Fix: strip them server-side before they reach the client and add <link rel="canonical"> as a safety net.

The canonical signal stack#

Google treats canonicalization signals as hints, not commands, and weighs them together. Its documentation ranks the ones you control as redirects, then rel=canonical, then sitemap inclusion; the rest add weight:

301 redirects: the strongest signal; the old URL drops out and signals move to the target
<link rel="canonical">: a strong hint; usually respected, but Google can override it when other signals disagree
Sitemap inclusion: listing only canonical URLs signals intent
Internal links: consistently linking to one URL version tells Google which one matters
Other signals: Google also prefers HTTPS over HTTP and weighs which version the web links to

The critical rule: all signals must agree. A canonical pointing to /page/ while all internal links use /page is a contradictory signal. Google will make a judgement call, possibly the wrong one.

Two 2025-2026 details: if you set the canonical with JavaScript, it must match the one in the HTML (or the HTML should have none), because Google reads it both before and after rendering (JavaScript SEO). And after you fix signals, Google "might hold pages in a duplicate cluster for up to two weeks", so wait before concluding a fix failed. Don't reach for noindex or the URL removal tool to force a canonical: both hide the page from Search entirely.

Cross-domain canonicals#

The rel="canonical" works across domains:

<!-- On partner-blog.com, which republished your article -->
<link rel="canonical" href="https://yoursite.com/original-article">

This is legitimate and Google generally honours it when the pages really are the same. What it won't honour: using cross-domain canonicals as a PageRank-extraction scheme to redirect authority from unrelated content. The relationship has to be genuine.

Near-duplicate content: harder but important#

Exact duplicates are easy. Near-duplicates (same template, marginally different data) are trickier.

The question to ask: does this variant provide genuinely different value to a user, or is it the same page with one word changed?

Product size/colour variants with identical descriptions → canonical to the parent product
Location pages where only the city name changes → consolidate or write genuinely local content
Date-archive pages → noindex; they're navigation, not content

For large e-commerce sites: estimate the combinatorial explosion of your facet system. Five filter groups with four options each, where a shopper can pick one option or none per group, already make 3,125 combinations (5^5) before sort orders. If your server returns a 200 for all of them, you have a crawl budget and duplication problem that needs architectural attention, not just canonical tags.

Running a duplicate content audit#

Crawl with Screaming Frog and run the Near Duplicate Content and Exact Duplicate Content reports
In Search Console → Indexing → Pages, check "Duplicate without user-selected canonical" and "Duplicate, Google chose different canonical than user": these tell you where Google is disagreeing with your signals
Compare your sitemap URL count to the indexed count in Search Console: large discrepancies often point to parameter duplication
For e-commerce, audit whether faceted navigation URLs are crawlable and assess the combinatorial scale

Related: Crawling & Indexing for robots.txt and sitemap strategy, Site Architecture for faceted navigation patterns that prevent duplication in the first place, and International SEO for why translations are not duplicates.

Next: AI Crawlers & Bot Access, deciding which AI bots may fetch your pages and what they may do with them.