The measurement changes when the quotation rule changes
The same digital novel can yield substantially different estimates of quotation-contained text without changing a single word of the input. In the two transcriptions examined here, the range across three fixed punctuation rules was 5.78 percentage points for Pride and Prejudice and 35.70 percentage points for Frankenstein. These differences concern a mechanical text measurement. They do not establish how much either author used dialogue, whether one narrative is more conversational, or whether readers prefer one style. The numerator depends on which tokens a rule places inside quotation spans, while the denominator depends on the selected transcription and extraction procedure.
The analysis began with a deliberately small check: the first fifty retained paragraphs of each book. All three rules returned identical percentages within each opening window. Extending the same rules to the complete extracted bodies exposed disagreements that the openings had not revealed. That sequence is central to the case study. Agreement in a short accessible passage was insufficient evidence that a quotation parser would behave consistently across the corresponding book. Neither the initial agreement nor the later disagreement establishes a universal property of novels; both are reproducible observations about two frozen inputs.
A bounded question rather than a literary ranking
The estimand is the percentage of defined word tokens classified as double-quotation-contained under a specified rule, within one extracted transcription. The sensitivity quantity is the maximum percentage minus the minimum percentage across the three rules for that same input. Keeping the denominator fixed within each comparison separates quotation-boundary choices from differences in book length. The analysis does not average the two novels into an English-fiction estimate. It does not sample authors, publication periods, countries, genres or readers, and it provides no confidence interval for an imagined population of books.
The sample is purposive and fixed: Project Gutenberg item 1342, Pride and Prejudice by Jane Austen, and item 42324, the 1831 version of Frankenstein by Mary Shelley. Conversation and epistolary or framed narration offer useful contrasting situations for examining quotation rules, but that selection rationale cannot support a general comparison of literary styles. No additional novel was added after the results appeared. The larger Gutenberg catalogue, language availability and format coverage are outside this study. Its contribution is a within-text measurement audit rather than another count of catalogue entries.
The books, their editions and the downloaded files
Original authorship remains with Austen and Shelley. Austen lived from 1775 to 1817; Shelley from 1797 to 1851. Pride and Prejudice first appeared in 1813, but the received item 1342 title matter identifies the 1894 George Allen illustrated source, with a preface by George Saintsbury and illustrations by Hugh Thomson. Saintsbury lived from 1845 to 1933 and Thomson from 1860 to 1920. Their contributions are identified rather than attributed to Austen, and the preface and illustration blocks are outside the measured narrative. This is not a comparison of first editions.
Item 42324 identifies H. Colburn and R. Bentley's 1831 publication, and its transcriber note describes a photo-reprint of that edition. The photo-reprint's own bibliographic identity is not supplied and is not invented here. The downloaded files were retrieved on October 30, 2026. The Austen text header gives a September 1, 2026 update, whereas its landing page reported September 29; the Shelley header gives October 23. These are distinct from historical edition dates. Recorded byte hashes, rather than a title or current download link alone, define the inputs. The punctuation is evidence about these transcriptions, not the authors' original manuscript typography.
Drawing and preserving the body boundaries
Both starts and ends were checked against the surrounding text. The Austen body begins with the opening universal-truth sentence after title matter, preface and contents. It ends with the final paragraph about the Gardiners, before the closing illustration, printer information and Gutenberg material. The Shelley body begins with Walton's first narrative sentence after the introduction and preface. Its initial letter addressee and date precede that anchor and are excluded. The body ends with the creature's disappearance before the end marker, printer information, transcriber corrections and license. Later letter dates, signatures and addressees remain inside the body.
Within those boundaries, complete illustration blocks are removed with a balanced-bracket procedure. Standalone chapter and letter headings are also removed. Blank-line separation defines paragraphs, and retained paragraphs are joined with two newline characters. The exact start and end anchors, normalized-source character offsets and hashes are supplied with the code. This is a reproducible extraction specification rather than a claim that every retained token is narrative prose. In-body footnotes and formatting artifacts not covered by these rules remain present, a limitation found by inspection and preserved rather than silently repaired after the full-body results.
Three rules fixed before the full-body comparison
Rule R1 is a global double-quote toggle. Curly double quotation marks are treated like straight double quotation marks, and every encountered mark switches the state between outside and inside. A completed pair defines a span even if it crosses paragraph boundaries. A final unmatched tail is excluded. This simple rule ignores the directional information carried by curly opening and closing marks. It also treats an opening mark repeated at the beginning of a continuing paragraph as a state change, even when a reader would understand it as continuation of the same quotation.
Rule R2 uses the same paired-toggle logic but resets at each paragraph. Completed pairs within a paragraph contribute tokens; an unfinished tail is excluded before the next paragraph begins. Rule R3 distinguishes curly opening from closing marks and carries quotation state across paragraphs. If quotation is already open, another opening mark at the beginning of a paragraph is treated as continuation. Straight marks otherwise toggle, with the same paragraph-start continuation convention. Unexpected internal curly reopening and stray closing marks are logged. These are three operational definitions, not a contest with an established correct answer. R3 remains a heuristic, especially for unusual typography and mixed quotation conventions.
Tokens, coverage and diagnostics
Words are matched by the declared expression [A-Za-z]+(?:['’][A-Za-z]+)*. Internal straight or curly apostrophes remain within a token when surrounded by alphabetic characters; hyphens split tokens and digits are excluded. Accented letters and ligatures are not covered by the ASCII alphabetic ranges and may therefore split words. A token enters the numerator when its midpoint falls inside a completed quotation span. This convention makes the computation exact and reproducible, but it should not be confused with a linguistically comprehensive tokenizer. Changing that tokenizer would define a different analysis and is not part of the reported comparison.
All three rules intentionally ignore single quotation marks as independent span delimiters. A single-quoted passage nested inside an already counted double quotation adds no extra coverage because its tokens are already inside the outer span. Top-level single-quoted speech, dash-delimited speech and unmarked reported speech can be missed. Diagnostic counts record completed spans, unmatched tails, continuation openings, stray closings and internal reopenings. For R2, an unmatched tail means that the toggle remains open at a paragraph end; it does not prove that the initiating glyph was semantically an opening quotation mark. Offset lists preserve where those tails occurred.
What the opening pilot showed and what it could not show
After corrected extraction, the first fifty retained Austen paragraphs contain 1,254 word tokens, of which 955 are counted by every rule: 76.1563 percent. The corresponding Shelley opening contains 3,885 tokens, with 156 counted by every rule: 4.0154 percent. There are 52 completed spans in the Austen opening and 13 in the Shelley opening, with no unmatched final tails under any rule. The windows contain different amounts of text because paragraphs vary in length. Their very different percentages should not be used to rank entire novels or characterize the authors' typical practice.
Passage-level inspection found a defect in the first extraction implementation before the pilot was accepted. A regular expression stopped illustration removal at a nested bracket, leaving bracket debris and a captionless illustration marker that consumed positions in the fifty-paragraph window. The invalid Austen result was 850 of 1,140 tokens, or 74.5614 percent. Balanced-bracket removal restored the intended window and changed the percentage by 1.5949 points. The invalid output is retained with an explicit label. It is evidence of an implementation failure, not a general estimate of extraction bias. Agreement among quotation rules had not detected that failure.
Complete-body sensitivity in the Austen transcription
The complete extracted Austen body contains 122,174 tokens across 2,060 retained paragraphs. R1 includes 57,181 tokens, or 46.8029 percent. R2 includes 50,280, or 41.1544 percent. R3 includes 57,345, or 46.9372 percent. The maximum-minus-minimum range is 5.7827 percentage points. R1 finds 1,751 completed spans; R2 and R3 each find 1,735. R2 records 32 unmatched paragraph tails, while R3 recognizes 32 continuation openings. R1 and R3 finish without an unmatched final tail; R3 also logs no stray curly closings or internal curly reopenings.
The proximity of the global-toggle and continuation-aware aggregate percentages does not establish that their span assignments are identical. Aggregated coverage can conceal compensating changes, and no token-level accuracy estimate is presented. The larger difference from the paragraph-reset rule shows that restricting completed pairs to a single paragraph excludes text included under the other definitions. The passage-level audit identifies multi-paragraph written letters among the relevant structures. Consequently, a larger mechanically included share cannot automatically be interpreted as recovering more spoken conversation. It may also include more of a letter or another quoted written object.
Complete-body sensitivity in the Shelley transcription
The complete extracted Shelley body contains 75,234 tokens across 768 retained paragraphs. R1 includes 38,625 tokens, or 51.3398 percent. R2 includes 11,769, or 15.6432 percent. R3 includes 30,545, or 40.6000 percent. The range is 35.6966 percentage points. R1 reports 388 completed spans; R2 and R3 report 293 each. R2 records 190 unmatched paragraph tails and R3 recognizes 190 continuation openings. Neither global rule ends with an unmatched tail. Because this file uses straight double marks in the measured text, R3's curly-specific stray-close and internal-reopen diagnostics do not resolve the underlying semantic questions.
These results demonstrate why a final even number of quotation marks is a weak quality check. A global toggle can return to the outside state at the end while having crossed inappropriate boundaries earlier. Repeated paragraph openings can change its state and alter long stretches of coverage without leaving a terminal imbalance. The audit confirms continuation structures in the creature's account and in a written letter, but it does not classify every disputed token or apportion the entire numerical difference to one feature. The observed range belongs to this transcription and these rules, not to Frankenstein in every edition.
A fixed systematic inspection across both bodies
Before computing the complete-body results, twelve paragraph positions per book were fixed using equally spaced indices from the first to the last retained paragraph. Specifically, position k is 1 + floor(k × (N − 1) / 11), for k from zero through eleven and N retained paragraphs. All twenty-four selected paragraphs were inspected. Their one-based indices, hashes and descriptive labels are available in the audit file. This procedure spreads inspection across the beginning, middle and end without selecting paragraphs because they produce the largest numerical disagreement.
The selected Shelley paragraphs include a continuing portion of the creature's account, single-quoted speech nested within that account, a continuation paragraph in Elizabeth's written letter, and text explicitly identified as an inscription. Another selected paragraph is an in-body footnote quoting a literary title. The Austen selections include direct speech, narration, indirectly reported questions and correspondence described within narration. Equal spacing does not guarantee coverage of every rare feature, and these are passage-level qualitative descriptions rather than an independently validated annotation set. The audit supports specific descriptions of encountered structures, not corpus-wide precision, recall, error prevalence or inter-rater reliability.
Why quotation-contained text cannot be renamed dialogue
A separate earlier purposive audit inspected twenty-six paragraphs, including twelve from the opening windows and fourteen later diagnostic passages. It records Austen letters with continuation marks, nested reported speech, a cited proverb and apostrophes. In Shelley it records an imagined recipient exclamation, literary references, spoken statements embedded in an unquoted letter, a reported question without quotation marks, and a multi-paragraph written letter. This audit is distinct from the fixed systematic set; some paragraphs overlap, so their totals must not be added as if every inspected passage were unique. Targeted examples are not used to estimate an error rate.
These categories matter because typographical quotation and communicative function answer different questions. A written letter may be fully enclosed in quotation marks without being a conversational exchange. An inscription can be quoted as an object encountered by a character. A phrase can cite a literary work, and imagined speech can represent an anticipated response rather than a spoken event. Conversely, a narrative may report that somebody asked a question without marking the words as a quotation. The present rules do not resolve those distinctions. Calling their outputs dialogue percentages would introduce a semantic claim that the measurement and audit have not validated.
Remaining extraction limits and the scope of the evidence
The fixed systematic audit exposed a Shelley footnote within the body, and the earlier targeted inspection found residual Austen formatting syntax around a letter salutation. Those tokens remain under the unchanged extraction specification. Retention makes the frozen calculation reproducible but prevents a claim that its denominator is purified narrative prose. Removing the footnote or formatting syntax would require a separately specified extraction variant and rerun, not an invisible correction to the present figures. Likewise, the ASCII tokenizer, treatment of paragraph boundaries and single-quotation exclusion are deliberate scope limits rather than features proved optimal by the observed results.
The sample is too small and too deliberately chosen to support a population conclusion. The historical texts are encountered through present-day transcriptions that may differ from source print books, and this study has not collated them against page images or manuscripts. No modern-versus-historical punctuation comparison is attempted. The work also lacks an independently annotated semantic gold standard. These restrictions do not erase the within-input sensitivity differences, but they delimit their meaning: another analyst can reproduce how these three procedures behave on these bytes, not infer an author's intention, a reader's experience or the correctness of every resulting span.
Prior work and the narrower methodological contribution
Literary quotation analysis is an established research area. The Project Dialogism Novel Corpus provides annotations for literary quotations and contextual attributes, while LitBank's quotation documentation distinguishes direct speech from other text wrapped in quotation marks. Therefore the observation that quotation marks can enclose something other than dialogue is not presented as a discovery. The present study does not replace those annotation resources or establish a better quotation-attribution model. Its three transparent rules were chosen to expose measurement decisions, not to compete with trained systems or to demonstrate state-of-the-art performance.
The narrower evidence is the combination of a frozen two-transcription comparison, a negative short-window pilot, a materially different complete-body result, explicit parser diagnostics and a documented extraction correction. It shows one way a reassuring preliminary check can be insufficient for the scope of a later calculation. The findings require independent verification of the code and interpretation; a separately declared editorial-material exclusion or semantic annotation exercise would address different remaining uncertainties. Adding more novels before those checks would increase volume without necessarily improving validity. No further corpus expansion is part of this analysis.
Chapter boundaries and rules for text within quotation marks are separate measurement choices; their effects do not establish pacing, dialogue prevalence or literary quality. When a closing frame changes the chapter-length distribution
Reuse, attribution and reproducibility
The two Gutenberg item pages identify the works as public domain in the United States, and the received headers and licenses were inspected without finding a restricted-work notice. Gutenberg's guidance permits research on unrestricted works but does not certify copyright status in every jurisdiction. The historical authors and contributors are long deceased; that fact supports cautious source selection without supplying universal legal clearance. Alisson Bento is responsible for this analysis, calculations and explanatory text, not for authorship or ownership of the novels, their historical preface or illustrations. No full novel or modern editorial addition is included in the proposed public downloads.
The companion files contain derived counts, audit labels and hashes, exact extraction methods, executable analysis code and source references. The licensed raw files remain separate and private. Retrieval used two individual plain-text files from a mirror listed by Gutenberg, without a recursive crawl, and requests were spaced by more than two seconds. A failed obsolete file path returned 404 before the supported generated path succeeded; no authorization restriction was bypassed. Dates of original publication, source edition, transcription update and retrieval are kept distinct. Reproducing the calculations does not itself authorize redistribution of source text, and jurisdiction-specific reuse requirements remain distinct from computational reproducibility.
How quotation rules change the measure
Complete extracted bodies · two exact Project Gutenberg transcriptions. QUOTATION-CONTAINED TOKEN SHARE — NOT DIALOGUE SHARE
Full chart and source
Scroll horizontally to inspect the chart. ↔
Chart and values
Scroll horizontally to see all columns.
| Transcription | Tokens | R1 global (%) | R2 paragraph (%) | R3 continuation (%) | Range (pp) |
|---|---|---|---|---|---|
| 1342 | 122174 | 46.8029 | 41.1544 | 46.9372 | 5.7827 |
| 42324 | 75234 | 51.3398 | 15.6432 | 40.6000 | 35.6966 |
Percent of declared word tokens mechanically inside matched double-quote spans. Not dialogue percentages.
Data & downloads
Scroll horizontally to see all columns.
| Transcription | Tokens | R1 global (%) | R2 paragraph (%) | R3 continuation (%) | Range (pp) |
|---|---|---|---|---|---|
| 1342 | 122174 | 46.8029 | 41.1544 | 46.9372 | 5.7827 |
| 42324 | 75234 | 51.3398 | 15.6432 | 40.6000 | 35.6966 |
Percent of declared word tokens mechanically inside matched double-quote spans. Not dialogue percentages.
Scroll horizontally to see all columns.
| Transcription | Rule | Included tokens | Matched spans | Unmatched tails | Continuation openings | Stray closes | Internal reopens |
|---|---|---|---|---|---|---|---|
| 1342 | R1 | 57181 | 1751 | 0 | 0 | 0 | 0 |
| 1342 | R2 | 50280 | 1735 | 32 | 0 | 0 | 0 |
| 1342 | R3 | 57345 | 1735 | 0 | 32 | 0 | 0 |
| 42324 | R1 | 38625 | 388 | 0 | 0 | 0 | 0 |
| 42324 | R2 | 11769 | 293 | 190 | 0 | 0 | 0 |
| 42324 | R3 | 30545 | 293 | 0 | 190 | 0 | 0 |
Zero final unmatched tails is not proof of correct segmentation. R2 tails are mechanical toggle residues.
Evidence & method
How we calculated it
- Two fixed English-original novels; exact source hashes, body anchors checked against surrounding text and tokenizer frozen before full-body comparison.
- Three unchanged quotation rules; denominator constant within transcription; all comparisons descriptive and deterministic.
- First fifty retained paragraphs preserved as exploratory pilot; invalid-v0 illustration-removal defect transparently corrected before extension.
- Twelve equally spaced paragraph positions per book inspected; prior purposive audit separately labeled; no semantic gold-standard accuracy claim.
What the data cannot tell us
- Purposive two-transcription sample; no author, genre, population, reader-preference or causal inference.
- Quoted letters, literary references, inscriptions and imagined speech are not uniformly spoken dialogue; top-level single/dash quotation excluded.
- Retained in-body footnote and formatting artifacts; ASCII tokenizer splits accented/ligature words; no manuscript-typography claim.
- No independent semantic annotation; zero final unmatched count does not prove correctness; R3 is heuristic.
- Jurisdiction outside USA not verified; reproducibility is distinct from source-text redistribution rights.
Original sources
- Project Gutenberg 1342: Pride and Prejudice
Project Gutenberg 1342: Pride and Prejudice: Jane Austen, 1775–1817; item marked public domain in USA. Downloaded text title matter identifies George Allen 1894, excluded Saintsbury preface and Thomson illustrations. Exact mirror byte hash in source manifest.
Accessed October 30, 2026
- Project Gutenberg 42324: Frankenstein
Project Gutenberg 42324: Frankenstein: Mary Shelley, 1797–1851; item marked public domain in USA; 1831 H. Colburn/R. Bentley version. File describes photo-reprint, bibliographic identity unspecified. Exact bytes frozen.
Accessed October 30, 2026
- Project Gutenberg License
Project Gutenberg License: Research reuse of unrestricted works; actual file license checked; outside-US jurisdiction not certified.
Accessed October 30, 2026
- Project Gutenberg Permission How-to
Project Gutenberg Permission How-to: Transcription/source-edition differences, acknowledgment linking and jurisdictional limits.
Accessed October 30, 2026
- Project Gutenberg Robot Access
Project Gutenberg Robot Access: No website crawl; supports designated acquisition procedures; example specifies two-second spacing.
Accessed October 30, 2026
- Project Gutenberg Official Mirrors
Project Gutenberg Official Mirrors: Lists pglaf mirror used for the two individual UTF-8 files.
Accessed October 30, 2026
- Project Gutenberg Mirroring How-to
Project Gutenberg Mirroring How-to: Supported mirroring guidance; rsync unavailable here; no recursive mirror performed.
Accessed October 30, 2026
- Project Dialogism Novel Corpus paper
Project Dialogism Novel Corpus paper: Vishnubhotla, Hammond and Hirst (2022): literary quotation annotation prior art; no corpus text downloaded or reused.
Accessed October 30, 2026
- LitBank official repository
LitBank official repository: Quotation layer distinguishes direct speech from other quote-wrapped text and addresses typography beyond quotation marks; prior-art context only.
Accessed October 30, 2026
- George Saintsbury author metadata
George Saintsbury author metadata: Author metadata establishes 1845–1933; Austen source preface contribution identified separately, not attributed to Austen.
Accessed October 30, 2026
- Bodleian John Johnson Collection: Hugh Thomson
Bodleian John Johnson Collection: Hugh Thomson: Institutional collection note identifies illustrator Hugh Thomson, 1860–1920; illustrations excluded.
Accessed October 30, 2026
- Jane Austen House: first reading of Pride and Prejudice
Jane Austen House: first reading of Pride and Prejudice: Historical original publication January 1813, distinct from analyzed 1894 source edition.
Accessed October 30, 2026
Cite & reuse this study
Alisson Bento. How quotation rules change the measurement in two novel transcriptions. Methodological analysis of Project Gutenberg transcriptions 1342 and 42324, retrieved October 30, 2026; historical source editions 1894 and 1831. https://alissonbento.com/research/quotation-measurement-sensitivity-two-transcriptions/
Analysis, calculations and explanation: Alisson Bento. Original novels: Jane Austen (1775–1817), Pride and Prejudice, and Mary Wollstonecraft Shelley (1797–1851), Frankenstein. Austen source preface: George Saintsbury (1845–1933); illustrations: Hugh Thomson (1860–1920), both excluded. Gutenberg items marked public domain in USA; actual licenses inspected; no universal jurisdiction clearance or endorsement claimed. Only derived summaries, code, methods and references provided; no full novels.
Corrections or editorial questions