Bulk URL dedupe and sort
Paste thousands of URLs: deduplicate them by meaning rather than by string, sort them, and filter by domain. Runs in chunks, so a huge list will not lock the tab.
Deduplicating by meaning, not by string
A string comparison keeps all four of these, which are one page:
https://example.com/a?b=1&a=2https://www.example.com/a/?a=2&b=1https://example.com/a?a=2&b=1&utm_source=newshttp://example.com/a#section
This tool compares them after normalization — tracking stripped, parameters sorted, www. and the trailing slash dropped, fragment removed — and keeps the first one it saw. The output is the original URL, not the normalized one, so nothing is silently rewritten.
Large lists
Processing happens in chunks between animation frames, so a fifty-thousand-line paste does not lock the tab. The progress readout is real. Everything runs locally; no part of your list is uploaded.
Lines that are not URLs
Blank lines are ignored, and lines that cannot be parsed are counted and skipped rather than passed through — so the output is always a clean list. The count tells you how many were dropped, which is usually enough to spot a paste that went wrong.
Related tools
- Bulk tracking parameter stripStrip tracking parameters across an entire list of URLs at once, with a count of what was removed from each and the option to include ambiguous parameters.
- URL normalizerPut a URL in canonical form: default ports, percent-encoding, trailing slashes, parameter order and www. Choose a lossless or a comparison-grade normalization.
- URL diffCompare two URLs component by component. Reordered query parameters are reported as a reorder, not as a wall of changes, and path segments are aligned.