The four things that go wrong
1. Soft breaks inside paragraphs
A soft break keeps the text in the same paragraph while moving it to a new line — the
thing you get from Shift+Enter in most editors. Pasted out, it frequently arrives as
U+2028. Browsers render it as a break, so the text looks fine and behaves badly, and no
pattern that names \n and \r will find it. Every mode here
recognises it.
2. Paragraph separators instead of blank lines
U+2029 does the same job as a blank line and looks nothing like one to a program. This is why "remove the blank lines" sometimes reports nothing to remove from a document that visibly has paragraphs: there are no blank lines, only separators. Keep paragraphs mode treats U+2029 as a break like any other, and the run-length rules do the rest.
3. No-break spaces
A very common invisible character in word-processor text, and the one that survives every attempt to collapse the spaces. The conversion is switched on for this page. The complete inventory of characters that look like spaces — twenty of them, with what every mechanism does to each — is on the extra-spaces page.
4. Zero-width characters
U+200B ZERO WIDTH SPACE and U+FEFF, which is named ZERO WIDTH NO-BREAK SPACE and is more often met as a byte-order mark. Both occupy no width whatsoever, so the only way to know they are there is a tool that draws them. Delete zero-width characters removes both; it is on for this page because word-processor paste is where they turn up.
Those four are not ranked. Nothing here counted how often each one turns up in real documents, because that would need the applications this page could not test — see below.
Why find-and-replace misses them: eight ways to mean "new line"
The Unicode Standard names 8 newline functions in section 5.8 of its core specification: LF, VT, FF, CR, CRLF, NEL, LS and PS. That list is the source of this table — read directly on unicode.org, where the core specification is published free. Every other column was produced by running the check in this build. None of it is recalled from memory about what JavaScript does.
| Character | Named in Unicode §5.8 | JS \s | Multiline ^ $ | split(/\r?\n/) | This tool splits |
|---|---|---|---|---|---|
| LF U+000A The line ending of everything that is not Windows. | yes | yes | yes | yes | yes |
| VT U+000B Line tabulation. Named by Unicode as a newline function; almost nothing on a PC treats it as one. | yes | yes | no | no | no |
| FF U+000C Form feed — historically "eject the page". Carries page structure, not line structure. | yes | yes | no | no | no |
| CR U+000D Alone, the classic Mac OS ending; in practice usually a CRLF that lost half of itself. | yes | yes | yes | no | yes |
| CRLF U+000D U+000A The Windows ending, and the one the internet protocols specify. | yes | yes | yes | yes | yes |
| NEL U+0085 The EBCDIC newline, surviving in mainframe exports. | yes | no | no | no | yes |
| LS U+2028 Unicode’s unambiguous soft line break. | yes | yes | yes | no | yes |
| PS U+2029 Unicode’s unambiguous paragraph break. | yes | yes | yes | no | yes |
| Recognised, out of 8 | 8 | 7 | 5 | 2 | 6 |
Read the bottom row first. The pattern in every "remove line
breaks" answer, split(/\r?\n/), recognises 2 of the
8: LF and CRLF. It misses VT, FF, CR, NEL, LS, PS —
which is to say it misses everything a word processor is likely to give you.
JavaScript does not even agree with itself. VT and FF are matched by
\s but are not line boundaries for a multiline ^ or
$, so a pattern built on one of those mechanisms behaves differently from a
pattern built on the other. And NEL goes the other way: it is a line boundary nowhere in
JavaScript and is not even whitespace, so no built-in mechanism will touch it at all.
This tool splits on 6 of the 8. It declines VT and FF on purpose: a form feed is a page boundary, and turning one into a line break would silently delete the fact that a page ended there. Both are shown in the reveal view with their own badges so the decision stays yours.
What we could not test, and what we did instead
Microsoft Word itself. We had no copy of Word to test against. So this page contains no table of "what Word puts in your clipboard when you press Shift+Enter", because we would have had to invent it. What is here instead is the part that can be established without Word: the Unicode definition of the character set, measured agreement between the mechanisms that process it, and a tool that names whatever your own paste actually contains. The about page sets out which numbers on this site are generated by running the tool, which were measured, and which are quoted from a document that was opened.
One Word paste, read badge by badge
The sample loaded above comes out as 7 lines — the counter under the box says so, because U+2028 and U+2029 each end a line just as an LF does. In it: a heading followed by a subtitle on the far side of a U+2028, a paragraph whose boundary is a U+2029 where a blank line would normally sit, one no-break space inside justified body text, and one zero-width space hiding at the end of a line.
Press Show invisible characters before doing anything. You will see an LS badge after the heading, a PS badge before the last paragraph, one NBSP badge inside the body text, and a ZWSP at a line end. That is the diagnosis. The conversion is then a formality — and, more usefully, you now know that the next document from the same source will have the same four problems.
Putting it back
Copy the result and paste it into the document with the "keep text only" paste option, or the plain-text paste shortcut, so the editor does not helpfully reintroduce formatting from the clipboard's rich-text flavour. If the text is going into a template that will re-wrap it anyway, prefer Remove all line breaks and let the destination do the wrapping — the difference is that the result carries one paragraph per paragraph, where leaving the breaks in would carry one line per line.