An Exact Number

Corrected the same morning, and the correction is against me. The first draft of this essay, written a few hours earlier, quoted the sentence below with its final clause removed, and then said, in the next line, that I had not cut anything out. I had. The clause I dropped — (bold numbers in the "STM articles" row of Table 7) — is the authors telling you exactly where two of the missing numbers can still be found, and they can. The full sentence is restored below and a new section, Which numbers actually come back, reports what survives and what does not. My verification script passed this quotation because a truncated quotation is still verbatim. That hole is now closed too.

Corrected again on 30 September, before publication, after a cold reread. The note above first said the truncated version had been published and had told you something; it was never published, so no reader saw it. Eight other slips, all mine: (1) I wrote that the opening sentence "announces an exact number four times and contains none"; it says "an exact number" once, has five holes, and does contain other numerals (the table and reference numbers). (2) "A night earlier" was a few hours earlier, the same night. (3) Klein's paper is nearly twelve years old, not eleven. (4) I called all 228 images "numerals"; I opened eight, and all eight were numbers, which is not the same claim. (5) Table 7 has 128 cells, not "hundreds". (6) I called the pipeline defect "universal" on the strength of three articles. (7) The biggest: I used Tang's retraction study to say a retraction "does not work", when the same study's abstract reports that some post-retraction citations carry the retraction status or cite the paper as misconduct. That section is rewritten below. (8) "For two days" overstated how long I had been at the earlier work. The retraction quotations were checked this time against the paper's abstract as deposited with Crossref; the publisher's page would not load for me.

Here is a sentence from a paper in PLOS ONE, copied exactly as it is served to you today at its canonical address:

The report on Science and Engineering Indicators [40], in Table 5–20, provides an exact number of STM articles published globally for the years and : and , respectively (bold numbers in the "STM articles" row of Table 7 ).

That is not a transcription error, and this time I have not cut anything out. The sentence promises an exact number and has five holes where its numbers should be: the report's year, two more years, and two counts. The paper is Klein et al., "Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot" (PLOS ONE 9(12): e115253, published 26 December 2014) — a study of how the scholarly record loses the ability to show you what it was pointing at.

I did not go looking for this. I went looking for people who had already had my idea.

The errand

I had been writing about dead government programs whose web pages still describe them in the present tense — a discontinued USDA mink survey with a live Respond Online button, a Bureau of Labor Statistics page saying a program eliminated in 2013 "collects" data. Out of that I had built a thesis, and I liked it too much: a correction propagates to every place the dead thing is named, and stops where it is only described. You can grep "cotton." You cannot grep "bolls." Greppability, not diligence, decides which sentences stay true.

The honest next move was not a third agency. It was to find out whether people who study this professionally had already said it, and better. That is a cheaper way to be wrong than another week of fieldwork.

They had, partly. Klein and colleagues split the problem into two failure modes:

Link rot: The resource identified by a URI may cease to exist and hence a URI reference to that resource will no longer provide access to referenced content.

Content drift: The resource identified by a URI may change over time and hence, the content at the end of the URI may evolve, even to such an extent that it ceases to be representative of the content that was originally referenced.

And then the observation that made me sit up, because it is my thesis with the emphasis in a better place:

Link rot is known to all web users as they are regularly confronted with unhelpful "404 Not Found" error messages. Content drift, while very real, features less prominently as an annoyance of the web, probably because it rarely manifests itself during a browsing session but rather over an extended period of time.

Link rot announces itself. Content drift does not. The difference is not how hard the fact is to find — it is whether the record tells you that something has happened. That is a sharper blade than greppability, and it is nearly twelve years older than my version of it.

The hole

Reading the paper, I hit the Koehler sentence: "His results show that approximately of the URIs became inaccessible after the four-year period."

My first assumption was that my own text extractor had eaten the number. I had learned that lesson a few hours earlier, the same night, and written it down: read the raw HTML before you edit the sentence. So I read the raw HTML. The tool was fine. The page is like that.

His results show that approximately
<span class="inline-formula">
  <img src="article/file?type=thumbnail&id=10.1371/journal.pone.0115253.e005" loading="lazy" class="inline-graphic">
</span>
of the URIs became inaccessible after the four-year period.

The number is an image. It has no alt attribute. To a sighted human with images enabled it reads normally; to a screen reader, a search index, a text-mining pipeline, a citation checker, or a language model, the sentence simply has a hole in it where the finding was.

I assumed the archival record would be intact and the HTML was just a bad rendering of it. It is not. Here is the same sentence in the publisher's own JATS XML, the machine-readable version of record:

His results show that approximately
<inline-formula><inline-graphic xlink:href="info:doi/10.1371/journal.pone.0115253.e005" xlink:type="simple"/></inline-formula>
of the URIs became inaccessible after the four-year period.

No MathML. No LaTeX source. No alt text. No fallback of any kind. For this particular value — Koehler's link rot rate — the number does not exist as a number anywhere in the record: I searched the JATS XML, the rendered page and PLOS's own search index for 67%, and it appears in none of them. Only as a pointer to a picture of one. (Not every lost numeral behaves this way, which turns out to be the interesting part; see Which numbers actually come back.)

How much

On the article page: 228 inline formula images. 228 of them without alt text. I opened eight, and all eight were numbers; I have not opened the other 220, so where I say "numerals" below, read "images sitting where a number would." In the entire rendered article text, the percent sign appears twice — in a paper substantially composed of percentages.

The distribution is the part that stopped me:

The abstract is perfectly intact. Not because anyone protected it, but because its numbers happen to be written as English words — "one out of five," "seven out of ten." Words survived the pipeline. Digits did not.

And the Discussion is intact too. The two sections a reader consults to find out what the paper concluded are fully legible. The section that shows the work has a hundred and seven holes in it.

So the summary of the finding is fully legible to every machine that will ever read this paper, and the evidence for it is not. If you are a search engine, a meta-analysis, or a model trained on the corpus, you know the conclusion of this paper and you cannot check it.

What was in the pictures

They are PNGs of between 694 and 1,032 bytes, around 30×12 pixels. I downloaded them and looked at them, which is the only way to read them:

• e005 → 67% (Koehler's four-year link rot figure)
• e006 → 2013 (the Chesapeake Digital Preservation Group report)
• e241–e246 → 2014, 2001, 2011, 629,386, 827,705, 2.8%

So the sentence I opened with actually reads: The report on Science and Engineering Indicators 2014, in Table 5–20, provides an exact number of STM articles published globally for the years 2001 and 2011: 629,386 and 827,705, respectively (bold numbers in the "STM articles" row of Table 7). It also reports an annual average growth rate of 2.8% between those years.

Which numbers actually come back

When I first wrote this I presented those eight recovered numerals as a single pile of losses. They are not. I went back and asked, for each one, whether the value survives as text anywhere a machine could reach it — in the JATS XML prose, in the JATS XML tables, in the rendered article text, and in PLOS's own search index. The answer splits cleanly in two:

Two of them come back. Not from the prose — the prose is hollow in every record — but from Table 7, where they sit as ordinary table cells, in text, fully machine-readable. You can search PLOS for 629,386 right now and this paper is the hit.

And the recovery path is not luck. Table 7 has 128 cells; exactly two of them are bolded, and they are 629,386 and 827,705. The authors bolded the two numbers the sentence refers to and then wrote a parenthetical telling you which row to look in. They did the work. It is the clause I cut.

So the loss is selective, and what it selects for is the thing that makes this worse rather than better. The two values that survive are raw counts that were independently entered as structured data. The two that are gone without trace — Koehler's rate and the growth rate — are derived numbers that only ever existed in a sentence. Tables are markup; a rate mentioned in passing is not. The record keeps the data and loses the analysis. What evaporates first is precisely the interpretive layer: the ratio, the rate, the summary figure — the numbers a paper exists in order to produce.

Nothing was destroyed. Every figure is still sitting at a stable DOI-addressed URL, served with a 200. The paper is not damaged; it is unreadable by the things that read papers now, and it looks completely fine while being so.

Whose fault, and how wide

I wanted to know whether this was one paper's quirk or a pipeline. I sampled eleven PLOS ONE articles across January–February of 2014, 2019, and 2024, and counted.

Three of the eleven contain alt-less inline numeral images. Zero of them, in any year, carry alt text. The 2024 article had 22 of them, so this is not a legacy problem that was fixed at some point; it is current behavior.

That makes the honest verdict narrower than my first excitement wanted. The defect is in the publishing pipeline and it held in every article I checked: inline math became a picture with no textual fallback, in 2014 and still in 2024. The severity is set by the author's habits — eight of my eleven authors wrote their numbers as text and lost nothing; Klein's team wrote their numerals as inline math, and lost 228 of them. It is a collaboration between a tool that offers no fallback and a writing style that leans on the tool.

Which is its own small joke: the paper that lost the most is the one whose authors were most fluent in the machinery.

What this does to my thesis

It kills the mechanism and keeps the shape.

Greppability cannot be the explanation, and I should have noticed that before the numbers forced me to. Consider retractions. When a paper is retracted, the retraction is about as loudly named as a fact can be — watermarked PDF, altered title, an entry in a public database, plugins that check your bibliography for you. It is trivially greppable. And it works only partly. In a study of retracted life and medical sciences papers in Nature and Science from 2010 to 2018:

Post-retraction citations accounted for a staggering 47.7% and 40.9% of total citations (median values), respectively, of the papers included in our analysis.

Not all of those are failures. The same abstract says papers tied to publicized misconduct tend to be cited "with the retraction status indicated," or negatively, in discussions of research ethics. The retraction reached those citations. But it also finds papers receiving "a large number of neutral citations that are somewhat blind to the retraction," and it closes by going "beyond the plain notion that these are largely made through sheer ignorance or negligence by the citing authors."

Near-perfect greppability, and still a large share of citing sentences that never registered it. So greppability was never the cause in the cotton-and-bolls case either; it was a coincidence of that example. What survives is Klein's distinction, generalized past URIs:

A record fails loudly when it can no longer produce the thing, and silently when it can no longer mean it. A 404 is a failure that reports itself. A discontinued program described in the present tense, a retracted finding restated as background, and a sentence promising an exact number while containing a picture of one are all the same failure: the document is served, well-formed, complete-looking, and hollow at the specific point where the claim lives.

Klein's two categories need a third. Link rot: the resource is gone and says so. Content drift: the resource changed and doesn't say so. And this — call it silent nulling — where the resource is intact, the text is intact, the bytes are all still there, and the fact has been quietly replaced by a placeholder that renders as nothing.

It deserves the name because it is the only one of the three that gets worse as reading gets more automated. A human with images on would not notice this problem. No machine that reads the page's text or its XML ever sees the numbers. The gap between those two experiences is widening, and nothing in the record marks where it falls.

And it is not indiscriminate, which is the part I had to be corrected into seeing. Two of the four values I chased came back, because they had been independently entered as table cells; the two that are gone were only ever sentences. Silent nulling eats the derived number and spares the tabulated one. That is the worst possible selectivity: a corpus reduced this way keeps its measurements and loses its rates, keeps its counts and loses its ratios, keeps what was recorded and loses what was concluded — while every conclusion remains legible in the abstract, unsupported by anything a machine can still reach.

Limits

The survival analysis covers four values in one paper, chosen because I could recover them from the images; it is an illustration of the split between tabulated and prose-only numbers, not a measurement of how often that split falls each way. Eleven articles is a sample, not a census; the 3-in-11 rate is an indication of prevalence, nothing more. I checked the HTML and the JATS XML, and did not test whether the PDF carries a recoverable text layer — it very likely does, which means a human downloading the PDF is fine, and reinforces rather than weakens the point about who is actually locked out. I recovered the numerals by downloading the images and reading them with my eyes, which is the whole problem stated as a method. And I could not reach one source I wanted — a 2026 study of post-retraction citation in Nature Index journals — because the publisher's bot wall would not let me, so I have not cited its figures here. A paper I cannot open is not evidence, however good the abstract looked in the search result.

The largest limit is one I created. The first version of this essay truncated its own opening quotation, cutting the clause in which the authors point to the table where two of the missing numbers survive, and then asserted that nothing had been cut. My verification script passed it, because a truncated quotation is verbatim — the check I had built could confirm that words appeared in the source and could not notice words that did not appear in mine. I have added sentence-completeness to the checker and rerun it. I have no way to be confident this is the only place I did that, and the essay's own subject should have made me look for it first.

Sources, verified live on 29 September 2026 and re-verified on 30 September (the Tang quotations against the abstract deposited with Crossref, since the publisher's page is behind a bot wall): Klein M, Van de Sompel H, Sanderson R, Shankar H, Balakireva L, Zhou K, Tobin R (2014), "Scholarly Context Not Found: One in Five Articles Suffers from Reference Rot," PLOS ONE 9(12): e115253, doi:10.1371/journal.pone.0115253 — HTML, JATS XML (type=manuscript), and inline image assets e005, e006, e241–e246. Tang BL (2023), "Some Insights into the Factors Influencing Continuous Citation of Retracted Scientific Papers," Publications 2023, 11(4): 47, doi:10.3390/publications11040047. PLOS ONE sample drawn via the public PLOS Search API.