How to deduplicate a list without losing order
When to keep first-seen order, how case and whitespace change what counts as a duplicate, and how line dedupe differs from sorting unique values.
Order-preserving dedupe versus sort-then-unique
There are two common ways to get a list of unique lines. The classic command-line approach sorts the text and then drops adjacent repeats, which is fast but scrambles your original order. This tool takes the other route: it walks the list top to bottom, remembers every line it has already seen, and keeps only the first appearance of each. The order you pasted is preserved, which matters when the sequence carries meaning, such as a ranked list, a changelog or steps in a process.
How case sensitivity changes the count
Whether two lines are duplicates depends on how strictly you compare them. With case sensitivity on, USA, Usa and usa are three separate lines. Turn it off and they collapse into one, with whichever spelling appeared first surviving. Choose case-sensitive matching when capitalisation is significant, like product codes or acronyms, and case-insensitive matching when you are cleaning free-form text where the same word may be typed several ways.
Why whitespace trips up naive dedupe
Lines that look identical often are not, because copied data hides trailing spaces, tabs or an accidental leading indent. A strict character-for-character comparison would treat 'apple' and 'apple ' as different and leave both behind. Trimming the ends before comparing solves this, so visually identical lines are recognised as the same. The tool trims only for the comparison, so the line it keeps still holds its original spacing if you need it later.
What line dedupe cannot do
This tool is deliberately line-oriented, so it cannot find duplicate words inside a paragraph or repeated cells in a comma-separated row. If your data lives on one line, split it onto separate lines first, remove the duplicates, then rejoin. It also does not fuzzy match, so 'colour' and 'color' or a typo and its correction remain distinct entries that you will need to reconcile by hand.