Skip to content
Sealpaste

Delete all recent work?

This removes the 0 items saved in this browser. Your settings stay.

Stop saving recent work?

New work won't be saved in this browser. The 0 items already saved can stay or go.

Remove Duplicate Lines

Lines

Paste lines, or

CtrlV paste · Ctrl↵ run now

Unique lines
Ready
Ln 1, Col 1
Runs in your browser. Nothing you paste is uploaded unless you share it.

Remove duplicate lines from a list, keep the first of each, ignore case or spaces, and sort A-Z or naturally

Paste a list and keep one of each line, in its original order or sorted, with counts of what was removed.

How to use it

  1. Paste the list, one item a line, or open a text, CSV or log file.
  2. Choose whether case and spaces count, and the order you want.
  3. Copy or download the unique lines.

Examples

A list of email addresses with repeats

Input
alice@example.com
bob@example.com
alice@example.com
carol@example.com
bob@example.com
Result
alice@example.com
bob@example.com
carol@example.com

Each address is kept where it first appeared; 2 duplicates removed.

Ignore case, then sort A → Z

Input
Banana
apple
BANANA
cherry
Apple
Result
apple
Banana
cherry

With Ignore case on, the first spelling met is the one kept.

Natural order for numbered names

Input
file10.txt
file2.txt
file1.txt
file2.txt
Result
file1.txt
file2.txt
file10.txt

A → Z would put file10 before file2; Natural reads the numbers.

What it keeps and what it drops

The first time a line appears it is kept, exactly where it was; every later copy of it is dropped. Nothing else in a line is changed, so a list of email addresses, IDs, URLs or log lines comes back in its own words, only shorter.

Lines break at a new line in any style: Windows (CRLF), Unix (LF) and old Mac (CR). If your text used Windows line breaks, the result does too. A final line break at the end of the text is not counted as an empty line.

Under the result you see how many lines were read, how many are left, and how many duplicates and empty lines went.

Which setting to pick

Ignore case: on, “Bob@example.com” and “bob@example.com” are the same line and the first one met is kept as written. Leave it off for IDs, hashes, passwords and anything else where case matters.

Trim spaces: on (the default), spaces and tabs at the start and end of each line are cut before comparing and in the result, so “ carol” and “carol” are one line. Turn it off when leading spaces mean something, as in indented code or YAML.

Drop empty lines: on (the default), blank lines are left out. Off, the first empty line is kept like any other line and later ones are duplicates of it.

OrderWhat it doesUse it for
Original orderKeeps lines where they first appearedLogs, steps, anything whose order matters
A → Z, Z → ADictionary order: apple, Banana, cherryNames, words, tags
Natural (2 before 10)Reads numbers in a line as numbers: file2, file10File names, versions, numbered items
Shortest firstBy length; lines of the same length keep their orderSpotting stray short or long entries

Questions

Which copy of a duplicate line is kept?
The first one. Later copies are removed and the kept line stays in its place, unless you choose a sorted order. With Ignore case on, the kept line keeps the case it was written in the first time.
How many lines can it handle?
The page stays responsive while it works because the work runs in the background of your browser. Files over 10 MB are not read.
Can it remove duplicate words instead of lines?
It compares whole lines. To dedupe words, put one word on each line first, for example with your editor’s find and replace (a space to a new line).
Is my list uploaded?
No. Everything runs in your browser, so your text stays on your device. Nothing you paste is uploaded unless you share a result, which is locked with your password first.

Related tools