Data Deduplicator
Remove duplicate rows from a CSV, matching on the whole row or one column.
Processing: This tool runs entirely in your browser. Your input and any file you open stay on your device — nothing is uploaded to a server.
How to use the data deduplicator
- Paste your CSV.
- Choose whole-row matching, or pick a key column.
- Check how many rows were removed, then copy the result.
About this tool
Two ways to define a duplicate, and the choice matters more than it sounds.
Matching on the whole row removes only rows that are identical in every field. That is the safe option: nothing is lost that differed in any way.
Matching on a single column removes rows sharing a key even when their other fields differ — the same email address appearing twice with different names, for instance. That is what you want when deduplicating a mailing list, and it is destructive in a way whole-row matching is not: the first occurrence is kept and the rest discarded, so if the later row held the more recent data, you have kept the wrong one. Sort the file first if recency matters.
The count of removed rows is shown, which is worth reading. A deduplication that removes far more than expected usually means the key column is not as unique as assumed.
Parsing handles quoted fields properly, so a comma, quote or line break inside a value does not split it. That is the difference between this and splitting on commas, and it is why exports containing addresses or free-text notes so often arrive mangled.
Everything runs in the page. Spreadsheets are usually the most sensitive files anyone handles — customer lists, payroll, financial records — so a tool that works on them without uploading them is doing the one thing that matters most here.
Common uses
- Cleaning a mailing list with repeated addresses.
- Removing rows duplicated by a bad export.
- Merging two exports without double-counting.
Frequently asked questions
- Which duplicate is kept?
- The first occurrence. If the later row holds fresher data, sort the file so the row you want appears first.
- Should I match the whole row or one column?
- Whole row is safe — it removes only exact repeats. A key column is what you need for a mailing list, but it discards rows whose other fields differ.
- Why were so many rows removed?
- Usually the key column is less unique than expected. Switch to whole-row matching to see how many rows are genuinely identical.
Related tools
Remove Duplicate Lines
Remove repeated lines from a list, with options to sort, trim whitespace and ignore case.
CSV Validator
Check a CSV for ragged rows, duplicate headers and structural problems before importing it.
Data Sorter
Sort CSV rows by any column, with numbers sorted numerically rather than as text.
Row Filter
Keep only the CSV rows matching a condition on one column.