The container, the grid, and the percent sign

MathPix asked the right question about their own defect: what box is this text in? This is that question answered on the same page, from both sides — including two defects of ours that the same exercise found.

arXiv 1804.10694v5 (Tiramisu), page 5 · round 6 of the MathPix correspondence · every number here is produced by out/lstscore.py, out/lstroundtrip.py and out/lstlang.py in the pdf2mmd tree, and the ground truth is the author’s own texsrc/overview.tex.

What they found

Our annotation schema requires every code line to sit inside a diagram box tagged algorithm or pseudo_code. Page 5 prints four listings. We typed all 77 lines as code and then drew three containers — and the GPU listing, the middle one in the right column, got none. Its 22 lines are orphans… That single missing box switches off everything code-specific we have.

Four behaviours are gated on that parent, including a guard whose whole job is to catch a code line that came back as maths. On that page it logged 182 lines and fired on none of them, because an orphan is not a code line as far as it is concerned. Their own scale figure: of 114,855 pages carrying at least one code line, 1,152 had no container anywhere, and 42.5% of those carry \mathrm against 4.2% of the pages that had one.

We draw no container

A listing is a monospace grid, and the block is wherever that grid runs. The measurement is the character cell: in NimbusMonL at 8pt the advance between two adjacent glyphs is 4.23pt, every gap is a whole number of those, and a line’s indent is round((x0 − left) / cell). Nothing has to be drawn and nothing has to be decided.

Where the author did draw a frame we read it, because a drawn boundary says one thing a glyph extent cannot — which glyphs belong to which block:

lst-027, frame=single
    frame        52.91 688.48 559.09 729.33
    glyph extent 48.32 688.28 566.86 739.09

Wider on the left because numbers=left sets the gutter outside the frame, taller because the caption sits above the box. A reader that takes the glyph extent takes the caption with it. But the frame is corroboration, not the container: over the 291 listings of our gold set, 134 draw no marks at all. If the block depended on a box, 44% of them would not exist.

Page 5, five listings

They drew three containers. We find five blocks, each with its own rectangle, scored line by line against the author’s own source:

Their hand-check of that page was 1 of 7 contained lines wrong and 9 of 13 orphaned. Ours on the orphan is 22 of 22 exact.

The five lines they quoted

The percent sign, and what we got wrong about it

There are 4 percent glyphs on that page, and every one of them is:

cid 37   glyphname percent    NimbusMonL-Regu      8.0pt
cid 37   glyphname percent    NimbusMonL-Regu      8.0pt
cid 37   glyphname percent    NimbusMonL-Regu      8.0pt
cid 37   glyphname percent    NimbusMonL-Regu      8.0pt

The font dictionary names the character — and for a reader that opens the text layer, that is the end of the question.

Correction, 2026-09-23. This section first said that reading it as : was “a decision taken without consulting evidence that is in the file”. That was wrong, and MathPix corrected it: their PDF pipeline never opens the text layer at all — every page is rasterised and read from pixels, a born-digital arXiv paper exactly like a scan. So nothing consulted the font dictionary and declined to believe it. % read as : is an ordinary glyph misread of ink, and the \% in the orphaned listing is the maths-mode habit, not a judgement against evidence. The sentence is withdrawn.

It also means 22 of 22 and 9 of 13 are not the same measurement, and the comparison above should be read with that in front of it. pdf2mmd reads the glyph stream, where the character’s identity is stated; MathPix reads the ink, where it has to be inferred. Those are different problems and the harder one is theirs. What survives is the product question — for a born-digital page the text layer is free ground truth, and cross-checking against it would have caught all four of the lines in the table above. MathPix have logged that as a suggestion and have not promised it.

Four defects of ours, from the same exercise

Their round-6 note says the question that pays is what box is this text in, not what did the model think this glyph was. Both of the following were that question, and we had not asked it.

A LaTeX macro inside a verbatim line

Our first reading of the orphan was

for(i1 in 0..\min  ((N-2)%32,32)+2)

We collapse a run of upright roman letters spelling an operator name — min, sup, lim — because TeX sets those as plain letters. It ran on typewriter text. TeX has no way to set an operator name in a monospace face, so three typewriter letters spelling min are three letters of code. It corrupted the spacing too: the merged glyph keeps the first letter’s x, so the grid then counted two cells of gap. This is their \mathrm, arrived at from the other direction — a code line that picked up maths habits.

A block that crossed the gutter

Every right-column line came back indented by 56 spaces, because left — the x of column 0 — came from the other column. The fix is not a threshold: inside one column the lines overlap, since a long line spans every other line’s indent, and between two columns nothing overlaps at all. Block 5 above is what remains of it: some lines arrive with glyphs from both columns already merged, upstream of the listing layer. 11 of 30, and our largest open defect on this page.

A glyph set 1.4pt low is not a line

listings does not set every character on its row’s baseline. The multiplication star of i0*32+i1 is placed 1.39pt low, and line grouping made it a line of its own — four times on this page, and the only thing wrong with our reading of the orphan before the fix:

base 676.88   int i = i0 32+i1
base 675.49   *

A block knows its row pitch — 7.17pt here — and nothing a fifth of a pitch away can be a separate row.

A box around two listings is not either listing’s frame

Caught only because we went to cut the crop above. The page-wide rules on this page band the figures, and each band holds two listings side by side; taking the smallest enclosing rectangle gave both of them the same rect and called both framed. That is the exact over-claim the frame was introduced to prevent, since the whole point of a drawn boundary is that it says which glyphs belong to which block. A frame is a listing’s only when it contains that listing and no other — which is why only block 5 above is marked framed, and why the crop is cut by a rectangle rather than by a rule someone drew.

The language is looked up, not guessed

Containers and languages are the same question. We run no language detector over the text. keywordstyle colours a word if and only if that word is in the keyword list of the language the author named, so the coloured words on a page are a subset of one language’s keywords — and listings ships 94 of those lists in lstlang1-3.sty. We read them out (22,553 keywords) and look the language up. It is the same move as reading mathabx.dcl for a glyph slot: ask the package that painted the page, not the model that looked at it.

It does not guess. The abstentions are honest: 26 pages colour no keywords at all, 22 are genuine ties — a C listing that colours only for and float is covered equally by Modula-2, Java and C — and 21 are pages where we found no listing.

Where the listing reader stands

Over the 291-file listing gold set, every number re-measured for this page:

Can the grid be seen from line geometry alone?

MathPix asked the one question in this exchange that could be answered with an experiment. Their mitigation for the orphaned-container defect is to treat a run of orphan code lines as an implicit code block, and its false-positive case is prose or display maths the model mistyped as code. The grid is the guard that case needs — but they do not read glyph advances. Per line they have a bounding box and a string.

The line-level stand-in is cell = width / characters, which in a monospace face is the same on every row and in a proportional face is not. The test is the spread of that ratio across a block. Measured over the 291 gold listings as positives and prose blocks from the math corpus as negatives:

A 25× separation in the medians, from the box and the string alone.

And the gutter is what breaks it

The first run of this measured the whole line with the line number in it. The number is a different font at a different size, so the ratio was never measuring one grid:

Excluding it moved the listing median by a factor of 18 and turned a statistic not worth having into a usable guard. A first result saying “this does not work” and a second saying “this works very well” differed only in whether the line-number column was dropped before taking the ratio. Anyone implementing it has to drop it, or measure nothing. The experiment is out/gridguard.py in pdf2mmd, with both numbers in its docstring so the negative one cannot be lost.

On their mitigation

Treating a run of orphan code lines as an implicit code block looks right from here, and it is close to what we do — except that we never needed the container, so there is no orphan state to recover from. If it helps the A/B: the signal that makes it safe for us is that the run is a grid. A false positive would have to be a block of prose whose glyph advances are all whole multiples of one cell, and prose is not set that way.

Withdrawn and superseded

Our round-6 report named the defect as an invented \mathrm. That was wrong and we withdraw it: nothing invents \mathrm, and the error is that those lines were in maths mode at all. We also did not report the corrupted parentheses in the GPUBlock line or the .. that became ..., both of which change what the code says.

On the harpoons: their re-run of 1205.5935v1 counts 62 pages of \rfloor, 54 of \downharpoonleft, 6 of 」 and 4 of \lrcorner across its 92 pages. That supersedes the 28 corrections in our residuals report, which stays published as it was.

pdfdrill · reports index · pdfdrill.github.io