Remove Duplicate Lines — Preserve Order, Optional Case-Insensitive

Real-world use cases

• Clean a mailing list before import

  You exported 4,200 subscriber emails from two sources and the CRM rejects duplicates. Paste the list, turn on trim whitespace and ignore case so "Bob@x.com" and "bob@x.com " collapse into one, then read the removed count. 380 dupes gone, 3,820 unique rows ready to import.

• Dedupe a keyword list for SEO

  After scraping 1,500 search queries across three tools, the same phrases repeat with different casing. Drop them in with ignore case on, keep first occurrence, and you get a clean unique set to feed into your content plan without 600 redundant rows skewing volume math.

• Collapse repeated log lines

  A 10,000-line error log spams the same three stack traces. Paste it, keep case-sensitive on (stack lines differ by a hex address you do not want merged), and skim the unique set to see you actually have 12 distinct failures, not 10,000 events worth of panic.

• Merge two shopping or task lists

  You and a roommate each wrote a grocery list and pasted them together. Turn on trim whitespace and ignore case so "Milk" and "milk " count once, dedupe, and the combined 40-item list shrinks to 31 real items with the first-typed order preserved so it still reads naturally.

Common pitfalls

• Leaving case-sensitive on when you meant to merge "Apple" and "apple". Flip ignore case first, or you keep both and wonder why dupes survived.

• Forgetting trim whitespace on pasted CSV cells. A trailing tab makes "id1\t" and "id1" look unique, so 200 real dupes slip through.

• Expecting sorted output. This keeps first-occurrence order on purpose; if you need A-Z, run the result through Text Sorter afterward.

Privacy