Similarity between two texts or two pages

How to use the tool

Paste a text into each box, or enter a page URL and click “Fetch”: the tool reads its text and, by default, keeps only the main content. An option adds the menus, header and footer, which repeat across a whole site and make any two pages look closer than they are. You can mix both, for example a live page in A and the draft of a new page in B.

Results update as you type. The figures come first, then the longest shared passages are listed, then both texts are shown with those passages highlighted, so you can see at a glance what repeats.

I mostly use it in two situations. On a site that has multiplied pages per city or per service from the same template, to find out what really changes from one page to the next. And before publishing, to check that a new page does not borrow too much from an existing page on the same site.

The three measures and what they tell you

Cosine on words compares how often each word appears in the two texts, regardless of order. It is 100% when both texts use the same words in the same proportions, and 0 when they share no word at all. It responds to vocabulary: two pages on the same topic, written separately, often get a high cosine because they talk about the same things with the same terms.

Jaccard on three-word sequences splits each text into runs of three consecutive words, then divides the number of shared runs by the number of distinct runs in both texts combined. I chose word sequences over character sequences for a practical reason: short runs of letters turn up in almost every text in a given language, so a character-based calculation gives a high similarity to two texts that only share a topic. Three words in the same order, on the other hand, point to a reused sentence or template. This measure therefore follows the same logic as the shared passages, on a finer scale.

Shared passages are runs of at least five words found identically in both texts. The tool shows how much of each text they cover, separately for A and B, because the two figures differ as soon as the texts are not the same length: a short page copied in full into a long one is 100% covered, while the long one is only partly covered.

For all three measures, capitals, accents and punctuation are ignored. Stop words such as “the”, “of” or “le” are left out of cosine and Jaccard by default, otherwise they would bring any two texts in the same language closer together. Shared passages always keep them, since a copied sentence contains them too. No overall score combines the three figures: each one answers a different question, and an average would hide what sets them apart.

What Google says about duplicate content

Google’s page on canonicalization starts on a reassuring note: duplicate content on a site is normal and does not break Google’s spam policies. Google groups pages whose main content is very similar and picks as canonical the one it considers the most complete and useful. The others remain duplicates and are crawled less often.¹ The guide to ranking systems also describes deduplication systems, which show only the most relevant results for a search.²

The phenomenon is old and widespread. In 2013, Matt Cutts, then at Google, estimated that 25 to 30% of the web’s content was duplicate, according to Search Engine Land’s report. He explained that Google does not treat it as spam, groups duplicates together and shows the best one, while reserving the right to act against excessive and manipulative duplication.³ In 2015, a Raven Tools study run with its Site Auditor across more than 200 million pages found duplicate content on 29% of sites, exact duplicate titles on 22% and duplicate meta descriptions on 17%, as reported by Search Engine Land.⁴

The ordinary risk of duplicate content therefore lies in which page gets chosen: if two of your pages look alike, Google will show one of them, not necessarily the one you prefer, and will visit the other less often.

Doorway pages: when similarity becomes a problem

Google’s spam policies target doorway pages, created to rank for specific searches rather than to serve visitors. Two of their examples describe pages that look alike: several pages or domain names that target specific regions or cities and funnel visitors to a single page, and substantially similar pages that sit closer to search results than to a clearly defined, easy-to-browse site structure.⁵

In 2019, Google’s John Mueller answered a site planning 1,300 city pages: “That sounds like doorway pages, not something I’d recommend,” Search Engine Roundtable reports.⁶

The same policies mention content produced at scale through automated transformations such as synonyms or translation.⁵ A series of pages where only the city name changes, or where a few words are swapped for synonyms, falls under this logic. The tool spots the first case well. For the second, see the limits below.

What similarity percentage is a problem?

Google gives no percentage above which two pages count as duplicates or doorway pages. The tool’s figures measure overlap; they deliver no verdict, and the tool does not invent one. The question to ask remains the one behind Google’s policies: does page B give its visitor something page A does not?

Some audit tools set their own rule. Semrush’s Site Audit, for example, flags pages that are 85% similar as duplicates, and notes that a shared header and footer on pages with little text can be enough to trigger the warning.⁷ That threshold is a convention of Semrush’s own tool. Its remark on headers and footers also shows why comparing the main content, as this tool does by default, gives a fairer picture.

In practice, I mostly look at the highlighted passages. If what repeats comes from the page template (a company presentation, guarantees, a form) and each page also has its own useful content, the similarity is normal. If the highlighted text covers almost the whole page and only a few names change, the two pages answer the same question.

What to do with two pages that are too close

Google leaves the method up to you: its page on consolidating duplicate URLs presents redirects and the rel="canonical" tag as strong signals, sitemap inclusion as a weak signal, and makes none of them mandatory.⁸ Depending on the case:

• Merge the two pages into a single, more complete one and redirect the old URL to it, when they answer the same search.
• Keep both and point to the main one with rel="canonical", when the second must stay available, for example a printable version or a filtered page.
• Rewrite the weaker page to give it content of its own, when it targets a genuinely distinct search: specific local information rather than a swapped city name.

How pages link to each other matters too: internal links show which page you consider the reference on a topic. My internal linking guide explains how to organise those links so the main page receives most of them. To check that a rewritten page does not overuse the same words, then run it through the keyword density checker.

The tool’s limits

The tool compares words and does not see synonyms: two texts that say the same thing in different words get low figures, even though Google’s policies list synonyms among automated transformations.⁵ Nor does it compare two languages with each other: according to Google, a translated page is only a duplicate if its main content stays in the same language and only the menus and footer change.¹

Reading a live page relies on the HTML returned by the server. A site that only displays its text after running JavaScript will look almost empty: in that case, copy the text from your browser and paste it into the box provided. Beyond 30,000 words per text, the rest is not compared.

When dozens of pages look alike, comparing them two by two soon reaches its limits. In an SEO audit, I compare all of a site’s pages to group those that overlap and decide, group by group, what to merge, redirect or rewrite.

Frequently asked questions