Paste a list, and this duplicate line remover keeps each line only the first time it appears and drops the repeats. The order of your list stays the same. Three tick boxes decide how strict the match is (ignore upper/lower case, ignore spaces at the start and end, remove empty lines), and the stats tell you lines in, lines out and how many duplicates were removed. Then copy the clean list, download it as a .txt file, or clear the box.
That is the short answer to "how do I remove duplicate lines". The longer answer is about one fiddly detail: two lines that look the same to you can be different to a computer.
"Apple" and "apple " are not the same line (until you say so)
Out of the box, the tool compares lines exactly, character by character. So these three are all different:
Appleapple(lower-case a)Apple(a space at the end you can't see)
Lists copied from Excel, Google Sheets, PDFs or WhatsApp often carry invisible trailing spaces and inconsistent capitals. That's why I put the two matching options right at the top:
- Ignore upper/lower case treats "Apple" and "apple" as the same line.
- Ignore spaces at start/end treats "Apple" and "Apple " as the same line.
- Remove empty lines drops blank lines from the result.
I chose exact matching as the starting point on purpose. Sometimes case really matters, for example product codes where "AB-12" and "ab-12" are different items, and I'd rather you switch on the loose matching knowingly than lose data silently.
Walking through one messy list
Say you are collecting names for a class trip and people sent them in a group chat. You paste these six lines (the fourth line has a space after "Patel"):
Rahul Sharma Priya Patel rahul sharma Priya Patel Amit Kumar Priya Patel
With no options ticked, mode "keep unique lines":
Rahul Sharma Priya Patel rahul sharma Priya Patel Amit Kumar
Stats: 6 lines in, 5 lines out, 1 duplicate removed. Only the exact repeat of "Priya Patel" went away.
With "ignore upper/lower case" and "ignore spaces at start/end" ticked:
Rahul Sharma Priya Patel Amit Kumar
Stats: 6 lines in, 3 lines out, 3 duplicates removed. Each person appears once, in the spot where they first showed up, written the way they were first written.
Switch the mode to "show only lines that were duplicated" (same options ticked) and Amit Kumar disappears, leaving only the names that appeared more than once. That mode is great for asking "who registered twice?" rather than "give me a clean list".
Lists people actually clean with it
| List | Settings that usually make sense |
|---|---|
| Email addresses | Ignore case and trim spaces. Email addresses are not case-sensitive in practice, so Ravi@Gmail.com and ravi@gmail.com are the same inbox. |
| SEO keyword lists | Ignore case and trim spaces, remove empty lines. Exports from different tools love trailing spaces. |
| Phone numbers | Trim spaces. Note that +91 98765 43210 and 9876543210 are still different lines, so make the format consistent first. |
| Student or member names | Ignore case and trim spaces, then check the "only duplicated" mode to see who signed up twice. |
| Product SKUs or codes | Often exact matching only, if your codes are case-sensitive. |
If you send a cleaned contact list to a mailing tool afterwards, please only message people who agreed to hear from you. A tidy list is not a reason to send more mail.
Keeping order vs sorting
The tool never reorders your list. The first appearance of each line stays exactly where it was, which matters for things like a queue, a sign-up order or a ranked keyword list.
If you want the clean list in alphabetical order as well, paste the result into the text sorter. It can also remove duplicates while sorting, but doing it here first lets you see the stats and check the "only duplicated" view before the order changes. For tidying messy spacing inside lines (double spaces between first and last name, for example), run the list through remove extra spaces before you dedupe, because this tool only ignores spaces at the start and end of a line, not in the middle.
What it can't catch
- Spaces in the middle. "Priya Patel" (two spaces) and "Priya Patel" stay different even with trimming on.
- Spelling variants. "Mohd. Asif" and "Mohammed Asif" are different text. The tool compares text, it doesn't guess that two names mean the same person.
- Duplicates within a line. It works line by line. If your items are separated by commas on one line, put each on its own line first.
- Spreadsheet columns. For a sheet where a duplicate depends on two columns together (name plus phone, say), Excel's or Google Sheets' own remove-duplicates feature is the better choice.
- Very large files. Everything runs in your browser, so a huge list of hundreds of thousands of lines may be slow on an older phone. A laptop handles big lists more comfortably.
Small tricks that save time
- Check the "duplicates removed" number before you copy. If it's zero but you expected some, the culprit is almost always case or trailing spaces, so tick those two options.
- Use download .txt for long lists instead of copy, so nothing gets cut off when pasting on mobile.
- Need a test list full of repeats to see how it works? The line repeater will make one in seconds.
- Want to know how many words the cleaned list has, for example for a vocabulary set? Paste it into the word counter.
Remove duplicates from a list, keep its order, and see exactly what changed. That is all this page does, and it does it inside your browser without sending your list anywhere.