Skip to content
TextArray
100% local

Remove duplicate rows by column

Drop CSV/TSV rows whose value in one chosen column already appeared.

Input
Output
Related tools
Remove duplicate linesText cleaning
Extract columnText manipulation
Change CSV delimiterDeveloper tools
Duplicate line finderText cleaning
Remove non-ASCII charactersnewText cleaning
Remove bracketsnewText cleaning
Remove duplicate rows by column
Whole-line deduplication misses the real problem: two rows are rarely identical, they just share the same e-mail, ID or SKU while everything else differs. This tool judges rows by one column of your choosing — the same key appearing twice removes the later row, keeping the data set one-row-per-key without a spreadsheet formula in sight.
Paste CSV, TSV or any delimited rows; the delimiter is detected from the first data line (comma, semicolon, tab or pipe) with a manual override for edge cases. Pick the key column by its 1-based number, tell the tool whether the first row is a header — headers are preserved and never compared — and choose which occurrence survives. Keep first is the default; keep last is the classic trick for changelogs and exports where the newest row per key sits at the bottom. Values are trimmed before comparison, and an ignore-case switch merges [email protected] with [email protected].
The tally reports the removals and the line count either side, so a paste that silently did nothing — wrong column, wrong delimiter — is immediately visible. Rows shorter than the key column count as having an empty value there and deduplicate among themselves.
One honest limitation: splitting is plain, not a full CSV parse, so a quoted field containing the delimiter itself will confuse the column count — run such files through the CSV delimiter changer first. Everything runs locally in your browser; your data never leaves your device.
FAQ
How is this different from removing duplicate lines?
Duplicate-line removal needs the entire row to match. Here only the chosen column is compared — two rows with the same customer ID but different timestamps count as duplicates, which is usually what a data set actually needs.
When should I keep the last occurrence instead of the first?
When the file is append-ordered and the newest record wins: exports where each update adds a row, logs, event streams. Keep last gives you the latest state per key; keep first preserves the original entry.
How does the automatic delimiter detection work?
The first data line is scanned for commas, semicolons, tabs and pipes, and the most frequent one wins. The tally shows which was used — if it guessed wrong, force the right one with the delimiter option.
What about quoted fields that contain commas?
The splitting is deliberately plain, so a quoted field with an embedded delimiter shifts the columns for that row. For such files, normalize with the CSV delimiter tool first (e.g. to tabs), then deduplicate here.
Is my data uploaded anywhere?
No. The deduplication runs entirely in your browser and your data never leaves your device.