Why the breaks are there in the first place
A PDF page does not contain lines. It contains instructions for putting glyphs at coordinates. When you select text and copy it, the viewer runs an extraction pass that has to infer where each line began and ended from those coordinates, and it marks every inference with a line break. The result is one break per visual row of the column — faithful to the page, useless as text.
A sourcing note, since this page is otherwise built on measurement. The paragraph above is background. Nothing in it is a quotation. The ISO 32000 specification is paywalled, and we did not read the free reprint Adobe publishes, so nothing here is presented as citing either of them. Everything below this heading is different: it is produced by running this tool over word lists printed in full on this page.
The hyphen problem, and why it has no clean answer
The hard part of a PDF paste is not the breaks. It is the hyphens. A justified column breaks long words across lines, and the fragments look like this:
informa-
tion → information (the hyphen was the wrap)
well-
known → well-known (the hyphen is part of the word)
Those two inputs are the same shape. A trailing hyphen after letters, a next line starting
with a lowercase letter. Nothing distinguishes them except knowing English, and even
knowing English is not always enough — re-cover and recover are
both words, and they mean different things.
So this tool does not offer accuracy. It offers three explicit settings, and a score for each of them over a corpus you can read.
| Setting | Rule |
|---|---|
| Off | Never touch a trailing hyphen. The two fragments are joined like any other pair of lines. |
| Simple | Delete the hyphen whenever a line ends with letters + hyphen and the next line starts with a lowercase letter. |
| Guarded | The same rule, except that the hyphen is kept when the fragment before it is on the compound-head list below. |
The scoreboard
Two corpora, 24 words each. Corpus A is genuinely hyphenated English compounds, where a wrap landing on the hyphen must leave the hyphen alone. Corpus B is ordinary words broken at a syllable, where the hyphen must go. Each of the 48 words was fed to the tool as two lines and the output compared against the correct spelling, character for character. Both corpora are printed in full further down — this is a score over a declared word list, not a percentage over a corpus nobody can see.
| Setting | Compounds kept intact | Split words rejoined | Correct overall |
|---|---|---|---|
| Off | 0 / 24 | 0 / 24 | 0 / 48 |
| Simple | 0 / 24 | 24 / 24 | 24 / 48 |
| Guarded | 15 / 24 | 22 / 24 | 37 / 48 |
Read it as three statements. Off scores 0 of 48 — not
because it is bad at hyphens but because it never joins the fragments at all, so
informa- and tion come out as
informa- tion with a space in the middle. Simple is perfect on
corpus B and scores zero on corpus A: it deletes every line-end hyphen, so every real
compound is destroyed. Guarded keeps the hyphen when the fragment before it
is on a list of common compound heads, which rescues 15 compounds and
costs 2 rejoins — international and understanding, whose first fragments
happen to be compound heads too.
And 9 of the 48 are wrong under every setting. They are:
twenty-one, mother-in-law, state-of-the-art, cost-effective, part-time, up-to-date, decision-making, user-friendly, x-ray. Each is a
compound whose first fragment is not a prefix anyone would list — you cannot fix
x-ray or mother-in-law with a prefix list, only with a dictionary.
That is the ceiling of the approach.
The compound-head list, in full
Guarded mode consults exactly 36 fragments. This is an editorial choice made for this site, not a standard, and it is the only reason Guarded beats Simple:
anti- · auto- · co- · counter- · cross- · e- · ex- · extra- · full- · half- · high- · inter- · long- · low- · mid- · multi- · non- · off- · on- · over- · post- · pre- · pro- · pseudo- · quasi- · re- · self- · semi- · short- · sub- · super- · trans- · ultra- · un- · under- · well-
Every word, and what each setting did to it
The full run. A tick means the output matched the correct spelling exactly; the cell shows what the tool actually returned.
| Input | Correct | Off | Simple | Guarded |
|---|---|---|---|---|
| well-⏎known | well-known | well- known ✗ | wellknown ✗ | well-known ✓ |
| well-⏎being | well-being | well- being ✗ | wellbeing ✗ | well-being ✓ |
| self-⏎esteem | self-esteem | self- esteem ✗ | selfesteem ✗ | self-esteem ✓ |
| twenty-⏎one | twenty-one | twenty- one ✗ | twentyone ✗ | twentyone ✗ |
| mother-⏎in-law | mother-in-law | mother- in-law ✗ | motherin-law ✗ | motherin-law ✗ |
| state-⏎of-the-art | state-of-the-art | state- of-the-art ✗ | stateof-the-art ✗ | stateof-the-art ✗ |
| long-⏎term | long-term | long- term ✗ | longterm ✗ | long-term ✓ |
| high-⏎quality | high-quality | high- quality ✗ | highquality ✗ | high-quality ✓ |
| cost-⏎effective | cost-effective | cost- effective ✗ | costeffective ✗ | costeffective ✗ |
| non-⏎profit | non-profit | non- profit ✗ | nonprofit ✗ | non-profit ✓ |
| e-⏎commerce | e-commerce | e- commerce ✗ | ecommerce ✗ | e-commerce ✓ |
| co-⏎author | co-author | co- author ✗ | coauthor ✗ | co-author ✓ |
| ex-⏎partner | ex-partner | ex- partner ✗ | expartner ✗ | ex-partner ✓ |
| over-⏎the-counter | over-the-counter | over- the-counter ✗ | overthe-counter ✗ | over-the-counter ✓ |
| part-⏎time | part-time | part- time ✗ | parttime ✗ | parttime ✗ |
| up-⏎to-date | up-to-date | up- to-date ✗ | upto-date ✗ | upto-date ✗ |
| decision-⏎making | decision-making | decision- making ✗ | decisionmaking ✗ | decisionmaking ✗ |
| off-⏎site | off-site | off- site ✗ | offsite ✗ | off-site ✓ |
| pre-⏎existing | pre-existing | pre- existing ✗ | preexisting ✗ | pre-existing ✓ |
| re-⏎enter | re-enter | re- enter ✗ | reenter ✗ | re-enter ✓ |
| semi-⏎final | semi-final | semi- final ✗ | semifinal ✗ | semi-final ✓ |
| sub-⏎committee | sub-committee | sub- committee ✗ | subcommittee ✗ | sub-committee ✓ |
| user-⏎friendly | user-friendly | user- friendly ✗ | userfriendly ✗ | userfriendly ✗ |
| x-⏎ray | x-ray | x- ray ✗ | xray ✗ | xray ✗ |
| informa-⏎tion | information | informa- tion ✗ | information ✓ | information ✓ |
| hyphen-⏎ation | hyphenation | hyphen- ation ✗ | hyphenation ✓ | hyphenation ✓ |
| docu-⏎ment | document | docu- ment ✗ | document ✓ | document ✓ |
| para-⏎graph | paragraph | para- graph ✗ | paragraph ✓ | paragraph ✓ |
| experi-⏎ment | experiment | experi- ment ✗ | experiment ✓ | experiment ✓ |
| gov-⏎ernment | government | gov- ernment ✗ | government ✓ | government ✓ |
| con-⏎tinuous | continuous | con- tinuous ✗ | continuous ✓ | continuous ✓ |
| extraordi-⏎nary | extraordinary | extraordi- nary ✗ | extraordinary ✓ | extraordinary ✓ |
| individ-⏎ual | individual | individ- ual ✗ | individual ✓ | individual ✓ |
| neces-⏎sary | necessary | neces- sary ✗ | necessary ✓ | necessary ✓ |
| oppor-⏎tunity | opportunity | oppor- tunity ✗ | opportunity ✓ | opportunity ✓ |
| particu-⏎larly | particularly | particu- larly ✗ | particularly ✓ | particularly ✓ |
| respon-⏎sible | responsible | respon- sible ✗ | responsible ✓ | responsible ✓ |
| signifi-⏎cant | significant | signifi- cant ✗ | significant ✓ | significant ✓ |
| tempera-⏎ture | temperature | tempera- ture ✗ | temperature ✓ | temperature ✓ |
| univer-⏎sity | university | univer- sity ✗ | university ✓ | university ✓ |
| appro-⏎priate | appropriate | appro- priate ✗ | appropriate ✓ | appropriate ✓ |
| calcu-⏎lation | calculation | calcu- lation ✗ | calculation ✓ | calculation ✓ |
| develop-⏎ment | development | develop- ment ✗ | development ✓ | development ✓ |
| environ-⏎mental | environmental | environ- mental ✗ | environmental ✓ | environmental ✓ |
| inter-⏎national | international | inter- national ✗ | international ✓ | inter-national ✗ |
| manage-⏎ment | management | manage- ment ✗ | management ✓ | management ✓ |
| organi-⏎sation | organisation | organi- sation ✗ | organisation ✓ | organisation ✓ |
| under-⏎standing | understanding | under- standing ✗ | understanding ✓ | under-standing ✗ |
The rest of the cleanup
No-break spaces
Justified text is set by stretching the spaces between words, and a lot of extractors emit U+00A0 for the stretched ones. The result looks like ordinary text with slightly wide gaps and behaves like nothing you can search. Press Show invisible characters and every one gets an NBSP badge. This page has the conversion switched on already; the full whitespace inventory is on the extra-spaces page.
Blank lines that are not blank
Extraction frequently leaves lines holding a single space or a single no-break space where the page had vertical white space. Keep paragraphs mode counts those as paragraph boundaries, which is usually what you want. If it is not — if you are getting a paragraph break every few lines — the culprit is a run of whitespace-only lines, and the blank-lines page has the eleven-way comparison of what counts as blank.
Ligatures and missing spaces
Some PDFs extract fi and fl as single ligature characters, and
some drop the space between words entirely where the font kerned tightly. Neither is a
line-break problem and neither is fixed here — but both are worth checking for before you
conclude that the break removal mangled something.
A checklist for a long extract
- Paste it and read the badge counts before changing anything. They tell you what you are dealing with.
- Delete running heads, page numbers and footnote markers first — no automatic rule can tell a page number from a short last line.
- Use Keep paragraphs, never Remove all, unless the extract really is one paragraph.
- Set hyphens to Guarded and then read the seams. On the corpus above, Guarded still leaves 11 of the 48 words needing a human.
- Switch on no-break space conversion, then confirm the result in the reveal view — your eye will not catch what is left.