pdfdrill
a semantic compiler for documents · drill-down extraction · provenance kept

Drill into a PDF
with the cheapest
sufficient command.

pdfdrill does not store documents. It compiles them — into a typed graph where every node keeps a line back to the evidence it came from: the page region it was read from, the route that read it, and the competing readings that disagreed. A flat CLI drives it, starting shallow, returning prose (never JSON), and remembering what it already knows in a sidecar beside each PDF.

MathPix · Perplexity · Gemma 4 offline structural path · no key needed runs in the Claude.ai sandbox
▶ drill a PDF now — opens in your browser ⎇ pdfdrill repository ↳ see the QC pipeline ◳ browse a live document model ♥ support pdfdrill

The button opens drillspace in a prebuilt GitHub Codespace — the drillui terminal comes up on a sample paper, nothing to install. Needs a GitHub account. Browse a live document model needs neither: it opens pdfdrill-audit.html, a TiddlyWiki holding five drilled documents, with each inspector and source PDF beside it under files/.

Drill-depth escalation over a shared sidecar. A PDF file enters at L0 - free (about 40-60 ms): size, pdfinfo, links, dests. Escalating rightwards: L1 - cheap (cached, milliseconds) with abstract, toc, fonts; L2 - work (about 1 s) with md, page N, drill; L3 - model (offline, from lines.json) with model, compare, report, tiddlers. Above, mathpix (keyed) and ocr (keyless) both feed lines.json into L3, and a competing-readings group holds snip, vision and latex as scored provenance columns. Every tier drops a dotted line into one shared sidecar, where each command records facts so a re-run answers from cache.
Start shallow, escalate only on demand; every tier records into one sidecar. No coordinates are authored: a structural spec is laid out automatically, then emitted by MetaPost as EPS, PDF, DVI and the SVG above, and separately as an interactive Cytoscape rendering. Every form comes from the same derived geometry, so the paper figure and this one cannot drift apart. All eight panels build the same way.

01 — what it is

A flat CLI over a typed document model.

pdfdrill returns prose, never JSON, and persists what it learns in a sidecar next to each PDF — so a re-run answers instantly and a higher-level command never repeats a lower one. It wraps the heavy extraction tools so an LLM can reach for the cheapest command that still answers the question, escalating to OCR and the full document model only on demand.

There are five ways to drive it: the CLI itself; a drillui browser terminal for chatting with one drilled document; a stdio MCP server and a Streamable-HTTP MCP endpoint that hand drill results back as clickable resources inside Claude Desktop, Claude Code, or the claude.ai web custom connector; and a batch driver that walks a list of URLs or arXiv ids shallow→deep. Underneath, three packages do the work:

src/pdfdrill

The flat CLI + sidecar state machine + capture layer. Wraps the heavy extraction tools so you never call them by hand. Every command returns one sentence of prose and records cumulative facts — re-runs return instantly.

→ the SKILL the web chatbot drives

src/docmodel

The typed Document: DocObjects with anchor-based Streams, Realizations across streams, Alignments between them, and Region geometry. MathPix lines.json is the only format that lets you compare a LaTeX string against the image MathPix actually rendered.

→ the between-call memory

src/docops

Operators over a Document: Mutators modify in place (dehyphenate, Stanza NLP), Projectors emit artifacts — plaintext, LLM-compact markdown, TiddlyWiki tiddlers, the comparison table, and the full formula report.

→ where artifacts come from
Design principle

Anchors are opaque identities, not positions — inserts and deletes in one stream never invalidate references elsewhere. Source streams stay immutable; modules only add objects, realizations, and alignments. The raw MathPix payload is always recoverable verbatim.

02 — the killer case

The link that no text stream can see.

A paper's page-1 text says only “Our code is available here.” — where here is a hyperlink with no separate anchor text. Plain extraction, MathPix Markdown, and a chatbot PDF upload all drop the annotation layer, so the URL simply does not exist in anything an LLM reads. pdfdrill links reads the annotation layer directly and surfaces it in ~50 ms.

~/papers — finding an anonymized code release on a NeurIPS submission
$ pdfdrill links 2605.12061.pdf            # ~0.06 s · pure `pdfinfo -url`
Found 1 code/data host among external links:
  https://anonymous.4open.science/r/Unified-Representation-A9D9/  (page 1, no visible anchor text)

$ pdfdrill urls 2605.12061.pdf             # ~6 s · scans every page to recover anchor text
Same link re-derived — only reach for this when you need the visible text.

$ # MathPix / Markdown / chatbot upload: the link is gone — annotation layer never consulted.
Reach for the cheapest sufficient tool

links answers “where is the code?” in 50 ms by reading the annotation layer. urls re-derives the same answer in ~6 s. A MathPix pass misses it entirely. Powerful ≠ right — escalate only when a cheaper command genuinely can’t answer.

Layer 1, the PDF document as it really is: a text layer of headings and paragraphs, a font layer that proves it is born-digital, and an annotation layer dropped by every naive reader, holding hyperlinks with no visible anchor text and named destinations used as jump targets. Beside them, a scanned page with no text layer that needs OCR. The sidecar below is empty - nothing recorded yet.
Layer 1 — the PDF as it really is: the annotation layer and named-destination markers a text stream can’t see.

03 — drill depth

Start shallow. Escalate only on demand.

Every command records facts in the sidecar, so a higher level never repeats a lower one. pdfdrill size is always step zero — free, ~40 ms, and it tells you whether the auto-extracted text is even enough.

L0FREE
size · pdfinfo · links · dests — metadata, text-layer vs. scanned, annotation-layer URLs, named anchors. No extraction.
~40–60 ms
L1CHEAP
abstract · toc · fonts — “what is this about / what are the sections / is there math?” Usually all you need.
cached, ~ms
L2WORK
md · page N · drill — real extraction. Full Markdown to the sidecar, a single page, or chained size→fonts→abstract→toc→md.
~1 s
L3MODEL
model · compare · report · tiddlers — the unified docmodel + QC artifacts. Offline from an existing lines.json.
offline
~/papers — “what is this paper about?” → two subprocess calls, no extraction
$ pdfdrill size paper.pdf
18-page PDF, 0.2 MB, letter, produced by xdvipdfmx, has a text layer, not encrypted.

$ pdfdrill abstract paper.pdf
Abstract:

  For a finite planar set P, let ν(P) be the number of …
Scan detection

size probes the text layer at L0: a born-digital PDF has extractable text on page 1 and fonts; a scan has neither. Page-1 char count is authoritative (a stray stamp font won’t flip an image PDF to “has text”). When there’s no text layer it says “NO text layer — scanned, OCR required” and sets needs_ocr.

Layer 2, the L0 shallow probe: reads metadata, extracts nothing. size, pdfinfo, links and dests run in about 50 ms with no extraction, and a page-1 character count decides text layer versus scan. Three facts land in the sidecar: size, links, dests.
Layer 0 — probe: size · fonts · links · dests, and text-layer-vs-scan, in ~50 ms before any extraction.

04 — when the text layer lies

Eight pages of knots, and three readings that disagree.

pst-knot-doc.pdf is the PSTricks knot-plotting manual: 8 A4 pages out of dvips + Ghostscript 8.70. It has a text layer and it is not a scan — size reports both correctly. It still defeats the cheap route, and the three ways of reading it disagree about what the document contains.

text layer 8 elements ocr --force 119 mathpix 79 knot diagrams resolved 0 · 0 · 24
The docmodel inspector's element tree for the same eight-page document under three extraction routes. Left: the born-digital text layer yields 8 elements, one Paragraph per page, each holding a whole page of run-together prose and LaTeX source. Middle: tesseract yields 119 elements, finely split but uniformly typed as Paragraph and Table. Right: MathPix yields 79 elements carrying Diagram, Formula, Picture and Toc types.
The same eight pages, three routes. Element counts are the inspector's own badge — every object that carries a region.

born-digital text layer

8 elements. pdfdrill model falls through to pdfminer.six when no MathPix key is present. It returns exactly one Paragraph per page — a whole page of prose and LaTeX examples run together in a single blob, with no headings, no tables, no equations. The math fonts are right there in the font layer (CMMI10) and nothing typed comes out. Not empty; structureless.

ocr --force

119 elements. A raster pass: pages rendered at 400 DPI, then tesseract 5.3.4. Cost scales with page count, so this is minutes on a long document, not a probe. It recovers real structure — 109 Paragraph, 2 Table, and 8 Equation regions. The equation regions are correctly placed; their text is garbled, because tesseract does not read math.

mathpix

79 elements. Fewer than OCR, and the only route that sees what the document is about: 24 Diagram, 9 Formula, 2 Picture, 1 Toc. Requires paid MathPix keys, so unlike the two above you cannot reproduce this column for free.

More elements is not better extraction

OCR returns 119 elements and MathPix 79 — and MathPix is the better model. The 119 are finely split but uniformly flat: Paragraph and Table, nothing else. The 79 carry Diagram, Formula, Picture and Toc. Counting nodes measures how hard a route chopped the page, not how much of the document it understood.

Why the knots are invisible to both free routes

The knot figures are vector — 293 curves and 57 lines across pages 1–6, drawn by PostScript. There is no picture to extract: pdfimages reports 492 entries and every one is a 1×1 stencil mask of 1 byte, a dvips painting artifact. A raster or blob extractor finds 492 single pixels and nothing else. Only classifying regions on the rendered page recovers them as figures, which is why the Diagram column is 0 · 0 · 24.

both free routes, verbatim — keyless, no account, reproducible
$ pdfdrill model pst-knot-doc.pdf
[main] ingested 174 lines across 8 pages
[ParagraphProcessor] {'paragraphs_created': 8}
…is math-bearing (math fonts: CMMI10) but was built from the born-digital
text layer (pdfminer.six) with no MathPix key — that route captures prose,
not typed equations, so this model has 8 Paragraph and 0 Equation.

$ pdfdrill ocr pst-knot-doc.pdf --force
Tesseract OCR (eng): 191 line(s) across 8 page(s) at 400 DPI
  typed: section_header=2, page_info=5, table=2, equation=8
  8 equation region(s) typed — their TEXT is garbled (tesseract can't read math)

$ pdfdrill inspect pst-knot-doc.pdf
Docmodel inspector: 119 elements over 8 page(s).

Open the three models and hover the element tree — each row highlights its own region on the page. make-pstknot.sh in this repo regenerates the first two from the URL above.

↳ text layer · 8 ↳ ocr --force · 119 ↳ mathpix · 79

05 — command catalogue

One flat verb per job.

Run as PYTHONPATH=src python3 -m pdfdrill <command> <pdf> [args], or via the installed pdfdrill console script. Click a command to see what it returns.

Beyond math papers

pdfdrill now reads scanned commercial mail, not just arXiv math. The scans & entities tab adds continuity (recover Seite N von M markers MathPix crops away), segment (split a shuffled bundle into ordered documents), entities (offline-validated IBAN / BIC / address / Aktenzeichen), qr (decode GiroCode payment QR), elements (a layout GNN), and semantic (a cross-document evidence graph) — all keyless and self-contained.

selected command — prose return

    

    

      Layer 3, structural extraction and the first real content. L1 cheap answers what the document is about using abstract, toc and fonts; L2 work does real extraction with md and page N; math is detected because math fonts are present. The sidecar gains cached markdown and the toc.
      
Layers 1–2 — structure: abstract, Markdown, facts, math and code-fences pulled from the page.

    

  







  

    
06 — the qc pipeline

    
LaTeX · KaTeX · the image MathPix actually rendered.

    
MathPix lines.json is the only format that ships, for each equation, both the recognized LaTeX and the CDN crop it was read from. pdfdrill puts them side by side — and adds competing readings (MathPix Snip, an LLM on the crop, the author’s own .tex) as extra columns, each scored against a normalized LaTeX form. Below is a real row from report --embed on arXiv 2004.05631v1, eq (1.1):


    

      
LaTeX sourceKaTeX renderMathPix CDN crop

      
        

          
            a(x):=\{y \in Y \mid R(x, y)=1\}
\quad b(y):=\{x \in X \mid R(x, y)=1\}
            
snip 1.00

            
tex  0.99

          		
          
          
            
              MathPix crop of eq (1.1)
              
            
          
        

      
    


    
Each crop’s <img> links to the full page it was cut from (page_url() strips the region query). --embed base64-inlines every crop so the artifact has no live-CDN dependency — ideal for a preview that won’t load remote images. The page click-through stays live even when embedded.


    

      
snip
pdfdrill snip re-OCRs each crop through MathPix Snip /v3/text → a candidate LaTeX + a per-line confidence that becomes a score signal.

      
vision / candidates
candidates exports the crops for an LLM to read (no API key); vision runs GPT-4o over them. Both attach as extra provenance columns.

      
tex (gold)
pdfdrill latex aligns the author’s e-print .tex to each MathPix equation — the gold reference versus OCR, preamble-expanded so TikZ/macros resolve.

    


    

      
Experimental · LaTeX render path

      
A test input path renders tables straight from LaTeX rather than KaTeX: latex → DVI → dvisvgm emits a font-faithful vector SVG that drops into the HTML table — useful for full tabular layouts KaTeX can’t express. It stays keyless: latex already ships in the Claude.ai sandbox and dvisvgm (3.2.1) is apt-installed by the bootstrap, so it runs there as readily as on a local machine.

    


    

      
Closed loop

      
Scoring flags shaky equations; escalate exports only those for a second reading; after ingest, relearn re-scores and reports resolved / improved / still-shaky. On 2605.12061: 9 flagged → 7 resolved, 1 retained (the hardest multi-line equation, correctly kept).

    

    

      Layer 5, drill and provenance: one equation read four ways and each scored - mathpix 1.00, snip 0.99, vision 0.97, and latex as the author's gold .tex. All four point at the equation crop, the image mathpix actually read. Below, escalate exports the shaky ones, ingest takes the second reading, relearn re-scores. The sidecar gains latex_prov and score_prov per equation.
      
Layer 5 — drill · provenance: mathpix · snip · vision · the author’s gold .tex, each a scored column.

    

  







  

    
07 — live from the model

    
Real equations, rendered in your browser.

    
These are pulled verbatim from the extracted model of arXiv 2004.05631v1 (At the Interface of Algebra and Statistics) — the Galois connection at the heart of the thesis. Source LaTeX shown beneath each render; nothing was hand-typed.

    

    

      Layer 7, the QC report: LaTeX source, the KaTeX render, and the page image it was read from, side by side with a score. Three rows of source, render and page crop score 1.00, 0.99 and 0.62, the last one flagged. Every provenance column in the sidecar feeds the table.
      
Layer 7 — QC · report: LaTeX │ KaTeX │ image per equation, with score chips and review flags.

    

  







  

    
08 — the document model

    
Every element is a typed, queryable node.

    
The unified docmodel resolves a paper into typed DocObjects. Projected to TiddlyWiki, each becomes one tiddler — equations carry their LaTeX, reference number, page, the MathPix crop, region geometry, and every competing reading as latex_<provenance> + score_<provenance>.


    

      

        
Tiddler types in one extracted thesis

        

          
1116 formula (inline)

          
254 equation (display)

          
574 paragraph

          
137 diagram

          
135 page

          
134 sidenote

          
71 listitem

          
57 section

          
13 footnote

          
12 table

          
— reference / bibentry

          
2618 tiddlers total

        

        
In-text (N) references rewrite to {{<eq>||FREF}} transclusions; citations link straight to their bibliographic tiddler by number or citekey. The persisted model.docmodel.json is the complex memory that survives between LLM/tool calls.

      

      

        
One equation tiddler, verbatim

        
{
  "title": "2004.05631v1_EQ0001_p012",
  "text":  "<$latex text={{!!latex}} displayMode=true />",
  "tags":  "equation 2004.05631v1",
  "kind":  "Equation",
  "latex": "a(x):=\{y \in Y \mid R(x,y)=1\} …",
  "refnum":          "1.1",
  "equation_number": "(1.1)",
  "page":   "012",
  "canonical_uri": "https://cdn.mathpix.com/cropped/…-012.jpg?…",
  "width": "994", "height": "67",
  "top_left_x": "259", "top_left_y": "666",
  "latex_tex":  "a(x):=\{y\in Y\mid R(x,y)=1\}…",
  "score_tex":  "0.992",
  "latex_snip": "a(x):=\{y \in Y \mid …\}",
  "score_snip": "1"
}

      

    


    

      
Beyond the tree — the semantic graph (CSP) preview

      
pdfdrill semantic lifts the model into a constraint graph: every extractor becomes a sensor emitting evidence, and entities — Company / Person / BankAccount on a letter, Symbol / Claim / Reference on a paper — accumulate it, across documents when you pass --store graph.json. The pass itself is reified: each reading is a Question node, each transform a first-class Transformation node, and a derived belief column carries a deterministic best-reading tiebreak that a compiler type-checks for grounding and consistency. A federated vocabnet layer (MSC · PhySH · GND · STW) backs pdfdrill classify, while stex / scikgtex project the graph back out to enriched LaTeX — glossary, Table-of-Symbols and index, or an ORKG-annotated PDF whose contribution metadata rides along as XMP/RDF.

    

    

      
Why TiddlyWiki

      
It’s a single self-contained HTML file — no server, perfect for GitHub Pages — that holds the whole graph and renders the LaTeX widgets live through the KaTeX plugin. The artifact below is exactly that: pdfdrill tiddlers output, built into a real wiki you can open and explore.

    

    

      Layer 4, the unified docmodel: flat extraction becomes a typed, queryable graph. A text stream of opaque anchors - paragraph, equation, section, table - sits above a region stream carrying page geometry, with dashed alignments linking each text node to its region. The sidecar holds docmodel.json, the between-call memory.
      
Layer 4 — the L3 model: typed nodes, text/region streams, alignments and region geometry → docmodel.json.

    

  







  

    
09 — claude.ai & claude code

    
Nothing for you to install — just ask Claude.

    
You don’t run any of this yourself. Make pdfdrill available once — attach the bundle in a Claude.ai chat, or install the skill in Claude Code — then ask in plain language. Claude carries the skill (SKILL.md + the 89-command manifest), so it knows which commands to run, and it reads that manual — not you. The package is pure Python under src/ with no pip step, which is why the sandbox (the extraction tools and pydantic already aboard) runs it as-is. Everything in the terminal below is what Claude does for you — not a checklist you type.


    

      
you ask in plain language — Claude loads the skill and runs pdfdrill for you


you ▸ “Here’s pdfdrill (attached). Install it and load its skill, then tell me
       whether this paper’s equations extracted cleanly.”

# Claude — now holding the skill — does the rest. You watch; you don’t type any of this:
$ tar xzf pdfdrill.tgz                        # unpack the bundle you attached
$ cat src/pdfdrill/skill/SKILL.md            # the skill IS Claude’s manual — it reads this, you don’t
$ PYTHONPATH=src python3 -m pdfdrill size   paper.pdf
$ PYTHONPATH=src python3 -m pdfdrill report paper.pdf --embed   # → self-contained formula-report.html

# In Claude Code the skill is installed once, so step one is simply your question.
# (the attachable bundle is one `git archive --format=tar.gz HEAD` of the repo.)

    


    

      
keys from the environment
mathpix, snip, bibfetch read MATHPIX_APP_ID/KEY & PERPLEXITY_API_KEY from env or a git-ignored .env — never committed, never asked for mid-task. Every command that needs them says so before it runs; MathPix pricing.

      
offline by default
The structural path — model, compare, report, tiddlers, folder, latex — runs entirely from an existing lines.json. No key, no network.

      
keyless OCR fallback
No MathPix? pdfdrill ocr renders pages → tesseract → a MathPix-compatible lines.json (--lang eng+equ for math, eng+deu for German), and model falls back to it automatically.

    


    

      
Keyless LLM via Claude preview · not fully tested

      
New: when there is no API key but pdfdrill is being driven by Claude — the Claude Code CLI locally, or Claude.ai in the sandbox — it routes the prompt-driven sub-tasks (vision, bibtex) to that Claude, using pdfdrill’s own prompts. The prompt is the knowledge: a vision crop returns as recompilable LaTeX, a BibTeX record web-searched correct — no OpenAI or Perplexity key required. pdfdrill llm <pdf> --runtime reports the transport it detected: cli (synchronous claude -p --output-format json) or sandbox (a deferred request file the agent answers, then re-run to ingest). The attached result is byte-identical to the keyed API path. But it is not free: delegating a whole document to the running Claude can take more wall-clock time and burn more tokens than a single keyed call — so for steady use, setting up MathPix pay-as-you-go is often cheaper and faster.

    


    

      
A first-class Claude skill

      
pdfdrill now ships as a Claude skill. A single commands.yaml manifest is the source of truth for the whole 89-command surface — --help, the SKILL.md tables and the external drillui TUI are all generated from it, and a CI gate fails on any drift. A declarative prerequisite planner (pdfdrill steps, --ensure) chains the right commands toward a goal, so the agent asks for an outcome and pdfdrill walks the path.

    


    

      
Network access needed for the sandbox

      
In the settings menu, the Claude.ai sandbox can be set to allow network access to all hosts. Only after that change are the external services (MathPix — mathpix, snip; ChatGPT / GPT-4o — vision; Perplexity — bibfetch; and Gemma), fetching a PDF by URL, and even apt-installing dvisvgm at bootstrap possible. The keyless offline path — model, compare, report, tiddlers, ocr — touches no host and needs no change.

    


    

      
Do not upload PDFs to Claude.ai

      
Attach a PDF in a Claude.ai chat and it is silently flattened to Markdown — math turns to Unicode and the annotation layer (every hyperlink with no visible anchor text) is discarded before pdfdrill ever sees it. So don't hand Claude the bare PDF. Do one of these instead: (1) share a URL to the PDF — pdfdrill fetches it byte-for-byte; or (2) put the PDF in a .zip and upload that — the archive sidesteps the auto-Markdown conversion, so the real bytes reach the tool.

      
If a PDF is already sitting in the sandbox, tell Claude to run pdfdrill on that file — not the converted text: pdfdrill model <uploaded.pdf>, then pdfdrill llmtext <uploaded.pdf>. The llmtext dump is written to the doc's drill folder as <name>.drill/<name>.llm.txt. pdfdrill does not feed that file into the chat on its own, so explicitly ask Claude to read the .llm.txt file from the drill folder before it answers.

    


    

      
Self-hosting? Runs on CoCalc.ai

      
Prefer your own environment over the Claude.ai sandbox? A one-shot bash cocalc-setup.sh brings the CoCalc standard image up to spec — sudo apt-get for poppler-utils, dvisvgm and the LaTeX toolchain, then bun, uv/uvx and the Python deps into your user dirs. The drillui ask-the-document web terminal then runs there via bun run tools/drillui_bridge.ts, reached through CoCalc's port-forward. One CoCalc quirk: the terminal's WebSocket must carry the proxy path — paste wss://<host>/<project-id>/server/8787/ws into its Connect box. Full walkthrough: COCALC.md.

    


    

      
drillspace — try it in your browser

      
Click, wait for the codespace, and the drillui terminal opens in your browser on a sample paper that is already drilled — nothing installed, no command to type. Ask it a question, or type a pdfdrill command name to run it on the open document.

      
The container is prebuilt, so the toolchain is already baked in rather than installed on launch. It needs a GitHub account and spends its free-tier quota.

      

        ⎇ open drillspace
        ↳ repository
      

    

  







  

    
10 — live artifacts

    
Open the real output.

    
All produced by pdfdrill and served straight from this repository — no build step, no server. A formula report puts every inline formula and display equation side by side as LaTeX source │ live KaTeX render │ the MathPix crop it was read from, each row linking back to its page; a comparison does the same one row per display equation, scored across provenances.

    


    
five documents in one wiki · arXiv · a PDF-only submission · a scanned patent · a German thesis audit

    

      
        
◳ tiddlywiki · 1388 tiddlers · 5 documents · 14 MB + files/

        
Five drills, one browsable audit

        
One TiddlyWiki holding five documents drilled by different routes — 2209.00445v3 with LaTeX-gold source (280 tiddlers), 1012.3259 as a PDF-only arXiv submission with no LaTeX (265), USRE41428, a scanned US reissue patent (76, all page images), kolbe2018hubbard, a German BA thesis carrying the author's own .tex (568), and RiemannianGD (157). Each carries its section tree, paragraphs, tables and figures as tiddlers, and its formula report inlined.

        
Where to start: it opens on its own index. Click any Inspector: tiddler — Inspector: USRE41428 is the scanned patent, the hardest of the five. Each inspector is a separate file under files/ and opens full-screen in a new tab rather than in a panel inside the wiki, so the element tree gets the whole window. The five source PDFs are there too.

        open the audit →
      
    


    
arXiv 0902.0431 · per-equation QC with an independent residual formula reports

    

      
        
⊞ formula reports · index · 57 of 1,219 equations · 3.6 MB PDF + ink.json

        
The equations MathPix doubts — and what the ink says

        
A standing index of published formula reports. The first, 0902.0431, is filtered to --max-conf 0.5: the 57 equations of 1,219 that MathPix itself rated below 0.5, each as LaTeX source │ render │ scan crop. Beside every row sits a second, independent reading — an inkdrill residual measured between the render and the scan, which knows nothing about the confidence value. The two disagree usefully: 11 of the 55 measured come back clean or noise, pixel-indistinguishable from the page. MathPix under-rates itself about one time in five at the bottom of its own scale.

        
Also on that page: Three things per equation, from one call — why the whole pipeline is built on lines.json: the LaTeX, the confidence, and the crop that makes the confidence checkable; how that format became the toolkit’s interchange layer, with pdfdrill ocr, inkdrill and the pdfminer route all emitting it; and the two things pixel recognition cannot claim — the annotation layer and content inside images — which are what the second instrument is for.

        open the reports index →
      
    


    
arXiv 2607.02234 · Purified OPSD: On-Policy Self-Distillation docmodel inspector

    

      
        
⛏ inspector · 15 pages · 185 elements

        
The segmentation, inspectable

        
A DevTools-style lens on the extracted model: the rendered page on the left with every DocObject drawn as a hover/click box, and a sidebar on the right — a DOM-like element tree plus an inspector pane that shows the selected element's type, page, region, LaTeX (live KaTeX), props, realizations and alignments — with a reading-order reflow tab alongside. Built keyless through the pdfminer.six route, so the equations sit in the document's own coordinate system: the page rasters are inlined and every crop is cut client-side from them — a single self-contained file that needs no server.

        open the inspector →
      
    


    
arXiv 2409.18839 · MinerU: An Open-Source Solution for Precise Document Content Extraction translation

    

      
        
⛏ inspector · 14 pages · 305 elements · 🇬🇧 EN → 🇧🇷 PT-BR

        
The same document, in two languages

        
A translation lives in the model, not beside it. Every text-bearing element carries both readings, so the inspector shows one flag button in its top bar — 🇬🇧 EN for the original, 🇧🇷 PT-BR for the translation — and switching redraws the reflow view, the element tree and the inspector pane at once. The page rasters, the regions and the element ids never move: only the language of the text changes, so a translated paragraph stays anchored to the pixels it was read from. It opens on the translation; switch the flag to read the English it came from. 118 TableCell · 101 Paragraph · 32 TableRow · 16 ListItem · 14 Page · 13 Formula · 10 Footnote · 6 Diagram · 5 Table · 2 Sidenote · 2 Abstract.

        
The second translation example runs the other way: the Kolbe 2018 Hubbard thesis below is German with an English reading, 🇩🇪 DE → 🇬🇧 EN, in the same flag control. The button only appears when a document has more than one language.

        open the inspector →
      
    


    
Transformer Circuits Thread · Verbalizable Representations Form a Global Workspace in Language Models distill reading view

    

      
        
◈ distill · 216 sections · 58 tables · 141 diagrams

        
The paper, rebuilt as a Distill-v2 article

        
An Anthropic interpretability write-up (a print from the Transformer Circuits Thread) run through pdfdrill distill --embed: the docmodel projected back into the Distill v2 reading layout — named-column grid, runtime table of contents, late-bound ?? figure/equation references, and hover cite/footnote popovers. Math renders live in KaTeX; the 58 structured tables come back as span-aware HTML tables (rowspan/colspan), figures inline. One self-contained file — no template JS, no server.

        open the reading view →
      
    


    
arXiv 2004.05631v1 · At the Interface of Algebra and Statistics

    

      
        
◳ tiddlywiki · 2618 tiddlers

        
The document model, browsable

        
A self-hosted TiddlyWiki with the KaTeX plugin carrying the complete extracted structure — sections, equations, formulas, sidenotes, footnotes, references. Click any equation to see its full record and live render.

        open the wiki →
      
      
        
⊞ formula report · 1116 + 254

        
The KaTeX × MathPix comparison

        
1116 inline formulas and 254 display equations from the algebra-and-statistics paper.

        open the report →
      
    


    
bibkey heimUFT · Burkhard Heim — Unified Field Theory (study notes; derived model published with permission)

    

      
        
◳ tiddlywiki · 2876 tiddlers

        
A 121-page model, browsable

        
The MBB Lectures (1976) & Metron calculations as a self-hosted TiddlyWiki — 367 sections, 312 equations, 1020 formulas, 15 tables. Gruvbox palette, KaTeX live render; every tiddler prefixed with the bibkey heimUFT.

        open the wiki →
      
      
        
⊞ formula report · 1020 + 312

        
The KaTeX × MathPix comparison

        
The largest of the three — 1020 formulas and 312 equations over 121 pages.

        open the report →
      
      
        
▦ comparison · 310 expressions

        
One row per equation

        
The pdfdrill compare table: each display equation as LaTeX │ KaTeX │ the MathPix image side by side, scored across provenances.

        open the comparison →
      
      
        
⤓ tiddler data · 2876 tiddlers

        
heimUFT.tiddlers.json

        
The tiddler set behind that wiki: the unified document model projected one node at a time, every title, tag and transclusion keyed to heimUFT.

        download the json →
      
    


    
bibkey kolbe2018hubbard · Kolbe (2018) — Korrelation im Hubbard-Modell (German BA thesis, published with the author's permission)

    

      
        
⛏ inspector · 42 pages · 618 elements · 🇩🇪 DE → 🇬🇧 EN

        
The thesis, inspectable

        
The current drill of the whole thesis in the DevTools-style lens: 238 Formula · 124 Paragraph · 106 Link · 61 Equation · 42 Page · 38 Section · 19 Diagram · 13 Picture · 8 Footnote · 5 ListItem · 3 Table · 2 List · 1 Toc. Built through MathPix with the author’s own LaTeX ingested, so each equation carries gold source beside the read. Hover the element tree to highlight its region on the page.

        open the inspector →
      
      
        
◳ tiddlywiki · 564 tiddlers

        
A German thesis, browsable

        
Alexander Kolbe's BA thesis (Universität zu Köln) as a self-hosted TiddlyWiki — 38 sections, 61 equations, 238 formulas, 13 figures, all images embedded. With the author's own .tex ingested as a gold tex provenance column.

        open the wiki →
      
      
        
◳ tiddlywiki · DeepL DE→EN

        
The same thesis, in English

        
The German wiki run through pdfdrill translate --from DE --to EN-US: 176 prose tiddlers DeepL-translated in place, the original German kept under org_text. Equations and figures are untouched.

        open the English wiki →
      
      
        
⊞ formula report · 238 + 61

        
The KaTeX × MathPix comparison

        
238 formulas and 61 equations, rebuilt from the current drill. The only one of the three with a fourth column to check against.

        open the report →
      
      
        
▦ comparison · 61 expressions

        
LaTeX vs the author's own .tex

        
The pdfdrill compare table: each of the 61 display equations as MathPix LaTeX │ KaTeX │ image, with the author's gold .tex as a scored extra column. Rebuilt from the current drill.

        open the comparison →
      
      
        
⤓ tiddler data · 584 tiddlers

        
kolbe2018hubbard.tiddlers.json

        
The same projection for the German thesis, keyed to kolbe2018hubbard, images base64-embedded rather than linked. Regenerated from the current drill — 584 tiddlers, 20 more than the wiki above was built from.

        download the json →
      
      
        
⤓ tiddler data · EN-US

        
kolbe2018hubbard.en-us.tiddlers.json

        
The DeepL-translated tiddler set — each prose field holds the English text, the German source preserved under org_<field>, translated tiddlers tagged translated.

        download the json →
      
    


    
pdfdrill.github.io · this page's own figures

    

      
        
⛏ inspector · 8 pages · 9 elements

        
The figures on this page, drilled

        
The eight panels above, concatenated and run through the keyless born-digital route — 17 objects, 9 streams, 81 alignments. Worth opening for what it gets wrong: eight pages become eight Paragraphs, one blob per page, because MetaPost emits positioned text runs with no paragraph structure. The same failure section 04 describes, reproduced on our own output. Regenerate with make -C figures docmodel.

        open the inspector →
      
    

    

      Layer 6, projection to a new document: one node becomes one tiddler. The typed graph from layer 4, holding paragraph and equation, feeds a projector - a docops Projector - which emits one tiddler record per DocObject, assembled into a TiddlyWiki as one self-contained HTML file. The sidecar is read back by the projector.
      
Layer 6 — project: one model node → one tiddler → a single self-contained browser/HTML wiki.