RepeatText
Writing & List Tools

Duplicate Line Remover

Paste a list, choose how strict the match should be, and keep only unique lines.

Runs on your device. Nothing is uploaded.
Show
Result
  1. Paste your list, one item per line.
  2. Tick ignore case, ignore spaces at start/end or remove empty lines if needed.
  3. Choose keep unique lines or show only duplicated lines.
  4. Check the stats, then copy or download the .txt file.

Paste a list, and this duplicate line remover keeps each line only the first time it appears and drops the repeats. The order of your list stays the same. Three tick boxes decide how strict the match is (ignore upper/lower case, ignore spaces at the start and end, remove empty lines), and the stats tell you lines in, lines out and how many duplicates were removed. Then copy the clean list, download it as a .txt file, or clear the box.

That is the short answer to "how do I remove duplicate lines". The longer answer is about one fiddly detail: two lines that look the same to you can be different to a computer.

"Apple" and "apple " are not the same line (until you say so)

Out of the box, the tool compares lines exactly, character by character. So these three are all different:

  • Apple
  • apple (lower-case a)
  • Apple (a space at the end you can't see)

Lists copied from Excel, Google Sheets, PDFs or WhatsApp often carry invisible trailing spaces and inconsistent capitals. That's why I put the two matching options right at the top:

  • Ignore upper/lower case treats "Apple" and "apple" as the same line.
  • Ignore spaces at start/end treats "Apple" and "Apple " as the same line.
  • Remove empty lines drops blank lines from the result.

I chose exact matching as the starting point on purpose. Sometimes case really matters, for example product codes where "AB-12" and "ab-12" are different items, and I'd rather you switch on the loose matching knowingly than lose data silently.

Walking through one messy list

Say you are collecting names for a class trip and people sent them in a group chat. You paste these six lines (the fourth line has a space after "Patel"):

Rahul Sharma
Priya Patel
rahul sharma
Priya Patel 
Amit Kumar
Priya Patel

With no options ticked, mode "keep unique lines":

Rahul Sharma
Priya Patel
rahul sharma
Priya Patel 
Amit Kumar

Stats: 6 lines in, 5 lines out, 1 duplicate removed. Only the exact repeat of "Priya Patel" went away.

With "ignore upper/lower case" and "ignore spaces at start/end" ticked:

Rahul Sharma
Priya Patel
Amit Kumar

Stats: 6 lines in, 3 lines out, 3 duplicates removed. Each person appears once, in the spot where they first showed up, written the way they were first written.

Switch the mode to "show only lines that were duplicated" (same options ticked) and Amit Kumar disappears, leaving only the names that appeared more than once. That mode is great for asking "who registered twice?" rather than "give me a clean list".

Lists people actually clean with it

ListSettings that usually make sense
Email addressesIgnore case and trim spaces. Email addresses are not case-sensitive in practice, so Ravi@Gmail.com and ravi@gmail.com are the same inbox.
SEO keyword listsIgnore case and trim spaces, remove empty lines. Exports from different tools love trailing spaces.
Phone numbersTrim spaces. Note that +91 98765 43210 and 9876543210 are still different lines, so make the format consistent first.
Student or member namesIgnore case and trim spaces, then check the "only duplicated" mode to see who signed up twice.
Product SKUs or codesOften exact matching only, if your codes are case-sensitive.

If you send a cleaned contact list to a mailing tool afterwards, please only message people who agreed to hear from you. A tidy list is not a reason to send more mail.

Keeping order vs sorting

The tool never reorders your list. The first appearance of each line stays exactly where it was, which matters for things like a queue, a sign-up order or a ranked keyword list.

If you want the clean list in alphabetical order as well, paste the result into the text sorter. It can also remove duplicates while sorting, but doing it here first lets you see the stats and check the "only duplicated" view before the order changes. For tidying messy spacing inside lines (double spaces between first and last name, for example), run the list through remove extra spaces before you dedupe, because this tool only ignores spaces at the start and end of a line, not in the middle.

What it can't catch

  • Spaces in the middle. "Priya Patel" (two spaces) and "Priya Patel" stay different even with trimming on.
  • Spelling variants. "Mohd. Asif" and "Mohammed Asif" are different text. The tool compares text, it doesn't guess that two names mean the same person.
  • Duplicates within a line. It works line by line. If your items are separated by commas on one line, put each on its own line first.
  • Spreadsheet columns. For a sheet where a duplicate depends on two columns together (name plus phone, say), Excel's or Google Sheets' own remove-duplicates feature is the better choice.
  • Very large files. Everything runs in your browser, so a huge list of hundreds of thousands of lines may be slow on an older phone. A laptop handles big lists more comfortably.

Small tricks that save time

  • Check the "duplicates removed" number before you copy. If it's zero but you expected some, the culprit is almost always case or trailing spaces, so tick those two options.
  • Use download .txt for long lists instead of copy, so nothing gets cut off when pasting on mobile.
  • Need a test list full of repeats to see how it works? The line repeater will make one in seconds.
  • Want to know how many words the cleaned list has, for example for a vocabulary set? Paste it into the word counter.

Remove duplicates from a list, keep its order, and see exactly what changed. That is all this page does, and it does it inside your browser without sending your list anywhere.

Questions people ask

How do I remove duplicate lines from a list?

Paste the list into the box with one item per line. In the default mode, the tool keeps each line the first time it appears and removes later repeats, without changing the order. The stats show lines in, lines out and duplicates removed. Then copy the result or download it as a .txt file.

Why are some duplicates not being removed?

Matching is exact unless you change it, so 'Apple', 'apple' and 'Apple ' with a trailing space count as different lines. Tick ignore upper/lower case and ignore spaces at start/end to treat them as the same. Spaces in the middle of a line still count, so fix double spaces first.

Does removing duplicates change the order of my list?

No. The first appearance of each line stays exactly where it was and only the later copies are removed. If you also want the list in alphabetical order, paste the cleaned result into the Text Sorter afterwards.

How can I see only the lines that were duplicated?

Switch the mode to show only lines that were duplicated. Lines that appeared just once disappear, leaving the entries that repeated. It is handy for finding people who registered twice or keywords that showed up in more than one export.

How do I dedupe an email list?

Paste one address per line, tick ignore upper/lower case and ignore spaces at start/end, and use keep unique lines. Email addresses are not case-sensitive in practice, so this catches the same address typed differently. Please only email people who agreed to hear from you.

More writing & list tools

Same family, different job.