OneTaskBy Tim King · Updated
Find Duplicate Text
Find Repeated Phrases, Duplicate Sentences and Overused Wording
OneTask is free, with no sign-up, and built by one person.
If this tool saved you time, a coffee helps keep new ones coming.
How the Find Duplicate Text works
Find duplicate text in an essay, report, web page or list. Choose phrases, sentences or lines and the tool lists every one that appears more than once, most repeated first, so you can spot copy-paste accidents and the wording you lean on too often.
How to find duplicate text
In Phrases mode the text is split into words and every run of n consecutive words, called an n-gram, is counted. A sentence of ten words holds eight three-word phrases, starting at words one to eight. Sentences mode splits at full stops, question marks and exclamation marks; Lines mode compares whole lines after trimming spaces. With case sensitivity off, "The" and "the" count as the same.
The formula for how many n-word phrases a text contains is words − n + 1, which is why short phrases repeat far more often than long ones by chance.
First example. "We need to act now. We need to plan. We need to talk." with three-word phrases finds "we need to" three times.
Second example. The same text with two-word phrases finds "we need" and "need to", each three times.
How many words should a phrase be? Three or four words is a good default: two-word phrases flag ordinary pairs like "of the", while six or more catches only sentences copied whole.
How many n-word phrases a text of a given length contains (words − n + 1)
| Words in text | 2-word phrases | 3-word phrases | 4-word phrases | 5-word phrases | 8-word phrases |
|---|---|---|---|---|---|
| 50 | 49 | 48 | 47 | 46 | 43 |
| 100 | 99 | 98 | 97 | 96 | 93 |
| 200 | 199 | 198 | 197 | 196 | 193 |
| 300 | 299 | 298 | 297 | 296 | 293 |
| 500 | 499 | 498 | 497 | 496 | 493 |
| 750 | 749 | 748 | 747 | 746 | 743 |
| 1,000 | 999 | 998 | 997 | 996 | 993 |
| 1,500 | 1,499 | 1,498 | 1,497 | 1,496 | 1,493 |
| 2,000 | 1,999 | 1,998 | 1,997 | 1,996 | 1,993 |
| 2,500 | 2,499 | 2,498 | 2,497 | 2,496 | 2,493 |
| 3,000 | 2,999 | 2,998 | 2,997 | 2,996 | 2,993 |
| 4,000 | 3,999 | 3,998 | 3,997 | 3,996 | 3,993 |
| 5,000 | 4,999 | 4,998 | 4,997 | 4,996 | 4,993 |
| 6,000 | 5,999 | 5,998 | 5,997 | 5,996 | 5,993 |
| 7,500 | 7,499 | 7,498 | 7,497 | 7,496 | 7,493 |
| 10,000 | 9,999 | 9,998 | 9,997 | 9,996 | 9,993 |
| 15,000 | 14,999 | 14,998 | 14,997 | 14,996 | 14,993 |
| 20,000 | 19,999 | 19,998 | 19,997 | 19,996 | 19,993 |
| 30,000 | 29,999 | 29,998 | 29,997 | 29,996 | 29,993 |
| 50,000 | 49,999 | 49,998 | 49,997 | 49,996 | 49,993 |
| 75,000 | 74,999 | 74,998 | 74,997 | 74,996 | 74,993 |
| 100,000 | 99,999 | 99,998 | 99,997 | 99,996 | 99,993 |
Tips and pitfalls
Raise the minimum count for long documents: in a 10,000-word report, common phrases appear a handful of times anyway. For plagiarism-style checks inside one document, use eight to twelve words. Sentences mode treats abbreviations such as "e.g." as sentence ends, so split sentences can show as short repeats.
FAQ
What is a phrase here?
A phrase is a run of consecutive words, from 2 to 12 long, sliding one word at a time through the text. Punctuation between words is ignored, so a phrase can cross a comma but is matched on its words alone.
Can it find duplicate lines in a list?
Yes. Choose Lines to list every line that appears more than once with its count. To delete the duplicates rather than list them, use the remove duplicate lines tool, which keeps the first copy of each line.
Why do common phrases like "in the" show up?
Because they really are repeated. Short phrases of everyday words repeat in all writing; set the phrase length to three or four words, or raise the minimum count, to focus on the wording you chose.
How many results does it show?
Results are sorted by how often they appear, most repeated first, and the table shows up to 500 items. The stats show how many distinct repeated items there are in total, even when the list is cut.
Is my document uploaded?
No. The counting runs in your browser and nothing you paste leaves your device, so unpublished drafts, coursework and confidential reports are safe to check.
See also: all Writing & Editing tools, Lists & Lines, Markdown & HTML, Ciphers & Alphabets.
Related tools
Duplicate Line Remover
Strip repeated lines from any list, with case and order options.
Keyword Density Checker
Word and phrase frequency with density percentages.
Word Counter
Words, six readability scores, keyword density and platform limits.
Readability Checker
Flesch reading ease, grade level and five other readability scores at once.
Text Similarity Checker
Compare two texts and score how similar they are, four ways.