Skip to content

Original experiment · six chosen language fixtures

When ebook search passes but extracted text changes: a six-language experiment

A PDF search can find a word that is missing from the file’s default text extract. A repair can recover that word and quietly remove a different, legitimate hyphen. In this controlled experiment, both happened across six deliberately chosen language fixtures. Authors therefore need two checks: can the release find the intended passage, and does the extracted passage still match the manuscript?

Published
Share
Share this page
EmailCite & reuse this study
Key findings as text
Reusable imagesWhen ebook search passes but extracted text changes: a six-language experiment — 6 / 6: Native PDF search · Long word. Original fixtures · selected cases, not population ratesDownload sharing image (JPG) ↓

Key findings

6 / 6

Native PDF search · Long word

Original fixtures · selected cases, not population rates

0 / 6

Dehyphenated extract · Exact full text

Every repaired output misses one real hyphen; 11 of 12 blocks match.

12 / 12

XHTML source control + EPUB source control

Original fixtures · selected cases, not population rates

The decision before the score

An author preparing a downloadable reading sample may reasonably test its title, a character name and a distinctive sentence. Finding them is useful. It establishes that those queries worked in that particular program. It does not establish that a reviewer, translator or reader will receive the same characters when copying the surrounding paragraph into another document. Search, extraction and canonical text comparison are different operations, even when they begin with the same PDF.

The practical question here is deliberately narrow. When a line-ending hyphen is removed to recover a broken word, can the workflow also preserve a hyphen that belongs to the word? The answer in the tested settings was no. Two repair profiles recovered all six layout-broken target words but also deleted the six legitimate compound hyphens when those compounds crossed a line. Default native PDF search recovered the layout-broken words while default extraction did not.

This result suggests a release check with two opposing examples rather than one reassuring success. Include a word that should be joined and a compound that should retain its hyphen. Keep the original manuscript as the reference for deciding which operation is correct. An automatic rule that removes every qualifying line-ending hyphen cannot infer the author’s intention from that punctuation alone.

What the fixtures contain

The experiment uses six newly written microstories, one each in English, Portuguese, Spanish, French, Italian and German. Each fixture has two miniature chapters, a short narrative, a footnote and a small two-column table represented as three row blocks. These texts were created for the experiment; no published novel, customer manuscript or protected retail ebook was uploaded or processed. They are testing material rather than a representative sample of commercial books.

Each language has an exact canonical reference with twelve ordered text blocks. The same canonical passages appear in PDF, XHTML and EPUB. The reference also contains a proof card with a decomposed accent, a composed accent, a literal ffi ligature, ordinary letters and a circled digit. Those examples make normalization choices inspectable. They are intentionally present in the source and are not treated as accidental corruption merely because another Unicode spelling exists.

The English pair is reconciliation and well-being. Portuguese uses reconciliação and guarda-chuva; Spanish reconciliación and teórico-práctico; French réconciliation and rendez-vous; Italian riconciliazione and italo-francese; German Wiedervereinigung and E-Mail. These are fixed diagnostic targets. Six successes or failures mean six selected fixtures, not an estimate of the percentage of books or readers affected in those languages.

Four paired PDF conditions

For each language, the control PDF wraps normal lines without splitting either target. A second condition splits the long target word and adds a display hyphen at the break. A third splits the existing compound immediately after its real hyphen. A fourth includes both changes. The resulting grid contains twenty-four one-page PDFs. The XHTML and EPUB controls retain the unbroken canonical source strings, with CSS allowing automatic hyphenation rather than inserting the display hyphen into their source text.

The design changes the specified line break and, for the long-word condition, the noncanonical display hyphen. The other passages, font sizes, block order and ordinary wrapping remain fixed. It does not compare polished commercial conversions or competing publishing applications. The PDF writer places text explicitly, making the intervention reproducible and its intended answer visible in the canonical reference. Breaking a word at a selected point is a diagnostic intervention, not a recommendation for typographically elegant syllable division.

The original plan was written before fixture generation and extraction. A pilot then exposed two construction problems: the serif font lacked the circled-digit glyph, and the first accent query could match the already composed example. The pilot is retained. Accepted fixtures add an openly licensed symbol-font fallback and a unique accent-pair query. The amendment is disclosed because this is a controlled, transparent experiment, not untouched preregistration or blind confirmatory analysis.

Tools and operations actually run

The offline run used PyMuPDF 2.0, pypdf 6.21.4 and lxml 6.1.3, with ReportLab 5.1.0 writing the fixtures. These were the releases returned by the official package index at the source review. PyMuPDF uses its default plain-text extraction and a second extraction profile enabling TEXT_DEHYPHENATE. Its default native search is measured separately. A native search result is not counted as an extracted-text result.

For pypdf, the run compares plain extraction, layout extraction and one declared postprocessing rule. That rule removes a hyphen immediately between a word character and a newline followed by another word character. It has no dictionary, language detector, manuscript lookup or contextual judgment. Its limitations are therefore part of the method rather than a surprise repair added after seeing which cases failed. The exact expression and full extracted strings are downloadable.

For XHTML, lxml traverses the body’s blocks, preserving inline text and tails. For EPUB, the script follows the package spine, then uses the same block extraction. Table cells receive explicit spaces and rows remain separate blocks. This is an authored extraction workflow using an XML library, not a claim that lxml is a complete EPUB reader. EPUB conformance and accessibility validation are separate questions and were not measured here.

Recovering one word can damage another

All six control PDFs matched their references after the declared whitespace folding in every PDF extraction profile. Both source controls also matched in all six languages: six XHTML and six EPUB fixtures. These successful controls matter because they show that the tested punctuation, accents, ligature, footnote and small table can survive these particular source and extraction paths when the diagnostic line breaks are absent.

With only the layout hyphen present, default PyMuPDF extraction, pypdf plain extraction and pypdf layout extraction recovered none of the six intact target words as exact substrings. PyMuPDF dehyphenation and the declared pypdf repair recovered all six. In that condition, those repair profiles also restored the complete whitespace-folded canonical text. This is the successful side of the intervention and should not be omitted merely because the compound condition creates a problem.

With both diagnostic breaks present, the repaired outputs again recovered the long word in six of six fixtures. They also changed well-being to wellbeing, guarda-chuva to guardachuva, and the corresponding compounds in the other four languages to unhyphenated strings. That is a measurable difference from the chosen reference. It does not require claiming that every resulting spelling is impossible or that every real text would be misunderstood. For quotation fidelity, changing the author’s exact punctuation is already the relevant failure.

Why an excellent aggregate can hide a failed quotation

The combined-break PDFs contain between 726 and 841 reference codepoints after whitespace folding. Default extracts differ by three edit operations; the repaired extracts differ by one. Expressing one minus the normalized edit distance as character agreement gives approximately 99.86% to 99.88% for the repaired outputs. That is a description of this exact comparison, not human accuracy, readability or an estimate for an ebook collection.

At the same time, none of the six repaired combined-break outputs equals its full canonical reference. Eleven of twelve reference blocks are recovered as exact substrings; the compound paragraph is not. A high whole-document agreement value and a failed passage can therefore coexist. The metric answers how much the strings differ overall. It cannot decide whether the changed character belongs to a title, a name, an instruction or a piece of punctuation that a quotation must retain.

Authors do not need to reject aggregate comparison. They need to supplement it with checks tied to the release’s actual use. A manuscript-wide difference report can locate unexpected changes. A few opposing sentinel passages can expose a repair that succeeds in one context and fails in another. Neither replaces editorial review, but both are more informative than treating a nearly perfect numerical average as proof that every passage is ready to copy.

Search is a separate success condition

PyMuPDF’s default native search found the unbroken long-word query in all six layout-hyphen PDFs. Its default extracted text contained that unbroken substring in none of them. The official documentation explains why these operations need not agree: search and plain-text extraction have different default flags, including dehyphenation. A positive search result is therefore compatible with a default text extract that retains a display hyphen and a line break.

The opposite diagnostic also matters. Native search did not find the exact canonical compound query in any of the six combined-break PDFs. The study records query presence, rather than trying to interpret highlighted rectangles as a human reading test. It does not test a browser’s PDF viewer, Acrobat, Kindle, Kobo or an operating-system clipboard. Their behavior may differ and needs its own observation before an author makes a claim about those programs.

For a release, keep the search check and add an extraction check. Record the application or library, version, query and export settings. If a reader reports that a search works but a copied quotation changes, those records help identify which operation produced the difference. They also prevent an attempted fix from being credited to the format generally when it actually belongs to a particular setting or postprocessing rule.

Normalize openly and keep the original

The experiment reports raw Unicode, NFC and NFKC comparisons separately. Raw does not mean byte-for-byte file equality: it means that Unicode whitespace is folded while the other source codepoints remain unchanged. NFC then normalizes both reference and output using canonical equivalence. NFKC additionally applies compatibility normalization. Python’s Unicode database version is retained with the environment so that the actual implementation is identifiable.

The proof card shows the distinction. Its first café uses a combining accent while the second uses a precomposed character. The typed composed-accent pair query fails in raw matching and succeeds under NFC. The expanded office pair and ordinary-digit pair need NFKC in this fixture because the source deliberately includes a ligature character and a circled digit. These are declared search equivalences, not permission to silently rewrite the canonical manuscript or every scholarly quotation.

Neither NFC nor NFKC fixes the diagnostic hyphen problem. A display hyphen followed by a line boundary and a legitimate lexical hyphen carry different authorial intentions, but normalization alone does not reconstruct those intentions. Keep an untouched original and label any normalized comparison copy. This makes it possible to explain whether a reported difference comes from extraction, a chosen equivalence rule or an actual source change.

An author’s release check and the study’s boundary

Before distributing a sample, select passages from the real final manuscript: an ordinary sentence, an accented name, a long word that wraps, a genuine hyphenated expression, and any table or note that readers may quote. Compare the exact exported passage with the manuscript. Test the word that should join and the hyphen that should remain. If automatic cleanup changes both, review those differences rather than applying the rule across the entire book without a reference.

Retain the source text and the release artifact together. When revising typography or conversion settings, repeat the same checks and keep the tool versions. A clean XHTML or EPUB source can provide an additional reference path, but this experiment does not prove that every ebook reader, conversion program or visual layout preserves the same copy behavior. Its twelve successful source controls describe only the files and block-aware extractor supplied here.

The downloadable records also make the denominator visible. Five PDF extraction profiles across twenty-four files produce 120 text outputs. Two structured-source profiles across six languages add twelve more. Comparing each of those 132 outputs under three normalization rules creates 396 metric rows. Those rows are repeated views of the same small set of artifacts, not 396 independent trials. Similarly, five substring probes per metric row produce 1,980 recorded checks without increasing the number of original books or languages. The forty-eight native search queries remain a separate table. Keeping these counts separate avoids making a compact diagnostic look larger or more representative simply because its results are recorded thoroughly.

PDF extraction benchmarks already exist. DocLayNet provides a large annotated layout dataset, and published comparisons evaluate multiple parsers across document categories. This study’s contribution is smaller and practical: paired original passages, opposing hyphen targets, disclosed Unicode rules and raw outputs that let an author reproduce a release check without sharing a private book. It offers no human reading certification, accessibility verdict, OCR comparison, causal sales result or search-ranking claim.

Search and extraction give different answers

PDF: both breaks · XHTML/EPUB: unbroken source controls

Full chart and source
Search and extraction give different answers. Every repaired output misses one real hyphen; 11 of 12 blocks match.

Scroll horizontally to inspect the chart. ↔

Chart and values

Scroll horizontally to see all columns.

Search and extraction give different answers
OperationLong wordReal compound
Default extract0 / 60 / 6
Native PDF search6 / 60 / 6
Dehyphenated extract6 / 60 / 6
pypdf + declared repair6 / 60 / 6
XHTML source control6 / 66 / 6
EPUB source control6 / 66 / 6

PDF: both breaks · XHTML/EPUB: unbroken source controls

Original fixtures · selected cases, not population rates

Download chart (SVG) ↓ · Download chart (PNG) ↓

A high agreement score still misses a passage

Repaired combined-break PDF · whitespace folded

Full chart and source
A high agreement score still misses a passage. Every repaired output misses one real hyphen; 11 of 12 blocks match.

Scroll horizontally to inspect the chart. ↔

Chart and values

Scroll horizontally to see all columns.

A high agreement score still misses a passage
LanguageCharacter agreementExact full text
EN99.8623%0 / 1
PT99.8661%0 / 1
ES99.8721%0 / 1
FR99.8705%0 / 1
IT99.8767%0 / 1
DE99.8811%0 / 1

Every repaired output misses one real hyphen; 11 of 12 blocks match.

Character agreement = 1 - normalized edit distance. Not human accuracy.

Download chart (SVG) ↓ · Download chart (PNG) ↓

Data & downloads

Scroll horizontally to see all columns.

Search and extraction give different answers
OperationLong wordReal compound
Default extract0 / 60 / 6
Native PDF search6 / 60 / 6
Dehyphenated extract6 / 60 / 6
pypdf + declared repair6 / 60 / 6
XHTML source control6 / 66 / 6
EPUB source control6 / 66 / 6

PDF: both breaks · XHTML/EPUB: unbroken source controls

Scroll horizontally to see all columns.

A high agreement score still misses a passage
LanguageCharacter agreementExact full text
EN99.8623%0 / 1
PT99.8661%0 / 1
ES99.8721%0 / 1
FR99.8705%0 / 1
IT99.8767%0 / 1
DE99.8811%0 / 1

Every repaired output misses one real hyphen; 11 of 12 blocks match.

Evidence & method

How we calculated it

  1. Selected synthetic fixtures are not representative books, readers or language prevalence. Two PDF engines and one shared structured-source extraction procedure were tested.
  2. A pilot font and accent-query correction is disclosed. Line breaks shift subsequent lines vertically. The 396 rows repeat 132 outputs under three normalizations.

What the data cannot tell us

  • Selected synthetic fixtures are not representative books, readers or language prevalence. Two PDF engines and one shared structured-source extraction procedure were tested.
  • Search presence, normalized string equality, human readability and accessibility are separate outcomes. No commercial reader clipboard, OCR, EPUB conformance, indexing, ranking or sales was tested.
  • A pilot font and accent-query correction is disclosed. Line breaks shift subsequent lines vertically. The 396 rows repeat 132 outputs under three normalizations.

Original sources

  1. DocLayNet document-layout dataset

    Established large layout dataset; background, not input to this original-fixture experiment.

    Accessed October 31, 2026

  2. Adhikari and Agarwal (2024), A Comparative Study of PDF Parsing Tools Across Diverse Document Categories, arXiv:2410.09871

    Prior comparison of ten PDF tools across six document categories; no claim that parser benchmarking is new.

    Accessed October 31, 2026

  3. Meuschke et al. (2023), A Benchmark of PDF Information Extraction Tools using a Multi-Task and Multi-Domain Evaluation Framework for Academic Documents, arXiv:2303.09957

    Existing academic-document multitask extraction benchmark.

    Accessed October 31, 2026

  4. PyMuPDF documentation: Constants and Enumerations (vars.html)

    Official definitions of dehyphenation and ligature-preservation flags.

    Accessed October 31, 2026

  5. PyMuPDF documentation: Appendix 1, Details on Text Extraction (app1.html)

    Official explanation of extraction order and different search defaults.

    Accessed October 31, 2026

  6. pypdf documentation: Extract Text from a PDF

    Official plain and layout extraction methods and ambiguities.

    Accessed October 31, 2026

  7. lxml documentation: The lxml.etree Tutorial

    Official XML traversal and text-tail handling used for the source controls.

    Accessed October 31, 2026

  8. Unicode Standard Annex #15: Unicode Normalization Forms

    Canonical and compatibility normalization are distinct; source text is retained.

    Accessed October 31, 2026

  9. PyMuPDF 2.0 release on PyPI

    Official pinned release and license metadata; exact installed version retained.

    Accessed October 31, 2026

  10. pypdf 6.21.4 release on PyPI

    Official pinned release and license metadata; exact installed version retained.

    Accessed October 31, 2026

  11. lxml 6.1.3 release on PyPI

    Official pinned release and license metadata; exact installed version retained.

    Accessed October 31, 2026

  12. ReportLab 5.1.0 release on PyPI

    Official pinned release and license metadata; exact installed version retained.

    Accessed October 31, 2026

Cite & reuse this study

Alisson Bento. 2026. When ebook search passes but extracted text changes: a six-language experiment. https://alissonbento.com/research/ebook-search-copy-text-preservation-six-languages/

Original fixtures, numerical results and figures credited to Alisson Bento; third-party tools and fonts retain their own licenses.

Corrections or editorial questions

Pass it on.

When ebook search passes but extracted text changes: a six-language experiment | Alisson Bentoalissonbento.com

WhatsAppFacebookXTelegramRedditThreadsEmailSMS