Twenty characters that look like a space
The list below is not typed out. It is produced by scanning every code point in the Basic
Multilingual Plane for the Unicode general category Zs — which finds
17 — and then adding the 3 characters that are equally invisible
in a text box without being Zs: CHARACTER TABULATION, ZERO WIDTH SPACE, ZERO WIDTH NO-BREAK SPACE. That
makes 20. Because the scan runs at build time, the table cannot be short by one,
and a future Unicode version that adds a space character would add a row here without anyone
editing this page.
Names come from two unicode.org sources read directly: the Unicode Utilities list for [:Zs:] on util.unicode.org, and NameAliases-17.0.0.txt for the control characters. Nothing in the name column is written from memory.
| Code point | Name | Zs | JS \s | trim() | / +/g | Tidy | Tidy, strong | Blank line |
|---|---|---|---|---|---|---|---|---|
| U+0009 | CHARACTER TABULATION | no | yes | yes | no | yes | yes | yes |
| U+0020 | SPACE | yes | yes | yes | yes | yes | yes | yes |
| U+00A0 | NO-BREAK SPACE | yes | yes | yes | no | yes | yes | yes |
| U+1680 | OGHAM SPACE MARK | yes | yes | yes | no | no | yes | yes |
| U+2000 | EN QUAD | yes | yes | yes | no | no | yes | yes |
| U+2001 | EM QUAD | yes | yes | yes | no | no | yes | yes |
| U+2002 | EN SPACE | yes | yes | yes | no | no | yes | yes |
| U+2003 | EM SPACE | yes | yes | yes | no | no | yes | yes |
| U+2004 | THREE-PER-EM SPACE | yes | yes | yes | no | no | yes | yes |
| U+2005 | FOUR-PER-EM SPACE | yes | yes | yes | no | no | yes | yes |
| U+2006 | SIX-PER-EM SPACE | yes | yes | yes | no | no | yes | yes |
| U+2007 | FIGURE SPACE | yes | yes | yes | no | no | yes | yes |
| U+2008 | PUNCTUATION SPACE | yes | yes | yes | no | no | yes | yes |
| U+2009 | THIN SPACE | yes | yes | yes | no | no | yes | yes |
| U+200A | HAIR SPACE | yes | yes | yes | no | no | yes | yes |
| U+200B | ZERO WIDTH SPACE | no | no | no | no | no | yes | yes |
| U+202F | NARROW NO-BREAK SPACE | yes | yes | yes | no | yes | yes | yes |
| U+205F | MEDIUM MATHEMATICAL SPACE | yes | yes | yes | no | no | yes | yes |
| U+3000 | IDEOGRAPHIC SPACE | yes | yes | yes | no | no | yes | yes |
| U+FEFF | ZERO WIDTH NO-BREAK SPACE | no | yes | yes | no | no | yes | yes |
| Handled, out of 20 | 17 | 19 | 19 | 1 | 4 | 20 | 20 |
Reading the bottom row
/ +/g handles 1 of 20. That is the
headline. The pattern everyone reaches for matches the plain space and nothing else, which
is fine right up until the text came from somewhere that uses a different one. The three you
will actually meet are CHARACTER TABULATION, NO-BREAK SPACE, NARROW NO-BREAK SPACE — a tab, a no-break
space, and its narrow cousin.
JavaScript's \s handles 19,
missing only ZERO WIDTH SPACE. It is broader than people
expect: it matches CHARACTER TABULATION and ZERO WIDTH NO-BREAK SPACE, neither of
which is a Zs character at all. That breadth is why /\s+/g is the
wrong tool for a different reason — it will happily eat your line breaks along with your
spaces, which is the fourth row of
the pitfall table on the front page.
trim() and \s agree on all 20 rows
, which is the one comforting thing on this page: if a pattern using \s matched
it, trim() will strip it from the ends, and vice versa. Both still miss the
zero-width space.
This tool handles 4 by default and 20 with both switches on, leaving nothing behind. The defaults are deliberately narrow: converting an EM SPACE or an IDEOGRAPHIC SPACE to a plain space changes how the text looks, and that should be a decision rather than a side effect.
One character worth its own paragraph
U+0085 NEXT LINE is absent from the table above: its job is breaking lines, not filling
them. But it belongs to this discussion because of what JavaScript does with it. Tested in this
build, /\s/ matches U+0085: false. It is
the one break character that \s does not cover, which means a "clean up all the
whitespace" pass built on \s is simultaneously too broad — it eats paragraph
separators — and too narrow. Naming the characters you mean is the only way out, and it is
what this tool does.
What the mode actually does, in order
- Zero-width characters are deleted, if that switch is on.
- Every
Zscharacter becomes U+0020, if that switch is on. - No-break spaces and narrow no-break spaces become U+0020.
- Tabs become a single space.
- Runs of two or more U+0020 collapse to one.
- Leading and trailing spaces and tabs come off each line.
The order matters and it is not arbitrary. Converting the exotic spaces before
collapsing means a run of EM SPACE, space, THIN SPACE becomes a single space.
Collapsing first would leave all three in place. Trimming last means a line whose only
content was a no-break space ends up genuinely empty, with no converted space left sitting
on it.
When not to run it
- Tab-separated data. Turning tabs into single spaces destroys the column structure irreversibly. Switch off Turn tabs into spaces first, or convert the file to CSV before touching it.
- Indented source code or YAML. Trimming and collapsing will destroy leading indentation, and in YAML that changes the meaning of the document rather than its appearance.
- Fixed-width or column-aligned reports. The alignment is the runs of spaces.
- CJK text with ideographic spaces. U+3000 is a normal typing key in Chinese, Japanese and Korean; converting it to a Latin space is a visible change to the text, not a cleanup.
The related jobs
Extra spaces usually arrive alongside the other two problems. Text out of a PDF has both wide gaps and hard wraps — do the line breaks first, because unwrapping creates new adjacencies that need collapsing afterwards. Text out of a word processor has spaces plus soft breaks and zero-width characters, which the word-processor page covers. And a line that holds nothing but whitespace is a blank line, which is a different mode with its own eleven-way comparison of what "blank" means.