Remove Duplicate Rows from CSV
Dedupe a CSV by the whole row or by a single key column. The match preserves quoting and respects the delimiter.
Quick answer: Dedupe a CSV by the whole row or by a single key column. The match preserves quoting and respects the delimiter.
Last updated
Frequently asked questions
- How do I dedupe a CSV by the whole row?
- Drop the file and leave every checkbox in the 'Match on' panel unchecked — that defaults to whole-row dedupe and keeps each unique row once. Enable 'Whitespace-insensitive match' if trailing spaces shouldn't count.
- How do I dedupe by one key column (like email)?
- Tick that single column in the 'Match on' panel. The first row containing each unique value is kept; later rows with the same value are dropped, even if their other columns differ.
- Which row is kept when there's a duplicate?
- The first occurrence in the file. If you want the latest record per key (e.g. the most recent purchase per customer), sort the file in reverse order before deduping, then re-sort after.
- Is the comparison case-sensitive?
- Yes — `Ada` and `ada` are treated as different values. Use the Find & Replace tool with `^.*` regex to lower-case a column first if you need case-insensitive matching.
- Does it preserve quoted fields?
- Yes. Quoting, embedded commas and newlines round-trip exactly through the parser.
- Will my CSV be uploaded?
- No. Deduplication runs entirely in your browser using Papaparse — nothing is sent to our server.
- Can I dedupe by multiple key columns?
- Yes. The 'Match on' panel lets you tick any combination of columns — leave them all unchecked to match on the whole row, or tick (e.g.) `email` + `signup_date` to dedupe on a composite key.
- What's the largest file I can dedupe?
- There's no fixed limit. A million-row file usually dedupes in a couple of seconds; the cap is your browser's memory.
- Why didn't a 'duplicate' row get removed?
- Almost always because of an invisible difference — trailing whitespace, a different quote style, or a Unicode look-alike character. Run the file through CSV File Cleaner with 'trim whitespace' on first.
- Does dedup preserve the original column order?
- Yes. Only rows are removed; the header and column order are untouched.