pdfdrill
drill-down extraction · pdf→latex ocr quality control

Drill into a PDF
with the cheapest
sufficient command.

A flat CLI that starts shallow, returns prose (never JSON), and remembers what it already knows in a sidecar next to each PDF — paired with a unified document model that takes a MathPixlines.json or an extended version of pdfminer.six as its input and turns it into a typed, queryable graph.

MathPix · Perplexity · Gemma 4 offline structural path · no key needed runs in the Claude.ai sandbox

The button opens drillspace in a prebuilt GitHub Codespace — the drillui terminal comes up on a sample paper, nothing to install. Needs a GitHub account.

Multi-layer drill-down process: original interface → initial extraction blocks → structural elements analysis → deeper semantic analysis → deeper drill layers → projection to a new document → final QA and HTML output
The drill-down pipeline, illustrated — start shallow, deepen only as needed, then project to a new document and emit clean HTML. (image generated with Gemini)

01 — what it is

A flat CLI over a typed document model.

pdfdrill returns prose, never JSON, and persists what it learns in a sidecar next to each PDF — so a re-run answers instantly and a higher-level command never repeats a lower one. It wraps the heavy extraction tools so an LLM can reach for the cheapest command that still answers the question, escalating to OCR and the full document model only on demand.

There are five ways to drive it: the CLI itself; a drillui browser terminal for chatting with one drilled document; a stdio MCP server and a Streamable-HTTP MCP endpoint that hand drill results back as clickable resources inside Claude Desktop, Claude Code, or the claude.ai web custom connector; and a batch driver that walks a list of URLs or arXiv ids shallow→deep. Underneath, three packages do the work:

src/pdfdrill

The flat CLI + sidecar state machine + capture layer. Wraps the heavy extraction tools so you never call them by hand. Every command returns one sentence of prose and records cumulative facts — re-runs return instantly.

→ the SKILL the web chatbot drives

src/docmodel

The typed Document: DocObjects with anchor-based Streams, Realizations across streams, Alignments between them, and Region geometry. MathPix lines.json is the only format that lets you compare a LaTeX string against the image MathPix actually rendered.

→ the between-call memory

src/docops

Operators over a Document: Mutators modify in place (dehyphenate, Stanza NLP), Projectors emit artifacts — plaintext, LLM-compact markdown, TiddlyWiki tiddlers, the comparison table, and the full formula report.

→ where artifacts come from
Design principle

Anchors are opaque identities, not positions — inserts and deletes in one stream never invalidate references elsewhere. Source streams stay immutable; modules only add objects, realizations, and alignments. The raw MathPix payload is always recoverable verbatim.

02 — the killer case

The link that no text stream can see.

A paper's page-1 text says only “Our code is available here.” — where here is a hyperlink with no separate anchor text. Plain extraction, MathPix Markdown, and a chatbot PDF upload all drop the annotation layer, so the URL simply does not exist in anything an LLM reads. pdfdrill links reads the annotation layer directly and surfaces it in ~50 ms.

~/papers — finding an anonymized code release on a NeurIPS submission
$ pdfdrill links 2605.12061.pdf            # ~0.06 s · pure `pdfinfo -url`
Found 1 code/data host among external links:
  https://anonymous.4open.science/r/Unified-Representation-A9D9/  (page 1, no visible anchor text)

$ pdfdrill urls 2605.12061.pdf             # ~6 s · scans every page to recover anchor text
Same link re-derived — only reach for this when you need the visible text.

$ # MathPix / Markdown / chatbot upload: the link is gone — annotation layer never consulted.
Reach for the cheapest sufficient tool

links answers “where is the code?” in 50 ms by reading the annotation layer. urls re-derives the same answer in ~6 s. A MathPix pass misses it entirely. Powerful ≠ right — escalate only when a cheaper command genuinely can’t answer.

Layer 1 — the PDF document: text, the annotation layer, and named-destination markers a plain text dump never sees.
Layer 1 — the PDF as it really is: the annotation layer and named-destination markers a text stream can’t see.

03 — drill depth

Start shallow. Escalate only on demand.

Every command records facts in the sidecar, so a higher level never repeats a lower one. pdfdrill size is always step zero — free, ~40 ms, and it tells you whether the auto-extracted text is even enough.

L0FREE
size · pdfinfo · links · destsmetadata, text-layer vs. scanned, annotation-layer URLs, named anchors. No extraction.
~40–60 ms
L1CHEAP
abstract · toc · fonts“what is this about / what are the sections / is there math?” Usually all you need.
cached, ~ms
L2WORK
md · page N · drillreal extraction. Full Markdown to the sidecar, a single page, or chained size→fonts→abstract→toc→md.
~1 s
L3MODEL
model · compare · report · tiddlersthe unified docmodel + QC artifacts. Offline from an existing lines.json.
offline
~/papers — “what is this paper about?” → two subprocess calls, no extraction
$ pdfdrill size paper.pdf
18-page PDF, 0.2 MB, letter, produced by xdvipdfmx, has a text layer, not encrypted.

$ pdfdrill abstract paper.pdf
Abstract:

  For a finite planar set P, let ν(P) be the number of …
Scan detection

size probes the text layer at L0: a born-digital PDF has extractable text on page 1 and fonts; a scan has neither. Page-1 char count is authoritative (a stray stamp font won’t flip an image PDF to “has text”). When there’s no text layer it says “NO text layer — scanned, OCR required” and sets needs_ocr.

Layer 0 probe: size, fonts, links and named destinations read in about 50 ms, before any extraction.
Layer 0 — probe: size · fonts · links · dests, and text-layer-vs-scan, in ~50 ms before any extraction.

04 — command catalogue

One flat verb per job.

Run as PYTHONPATH=src python3 -m pdfdrill <command> <pdf> [args], or via the installed pdfdrill console script. Click a command to see what it returns.

New — beyond math papers

pdfdrill now reads scanned commercial mail, not just arXiv math. The scans & entities tab adds continuity (recover Seite N von M markers MathPix crops away), segment (split a shuffled bundle into ordered documents), entities (offline-validated IBAN / BIC / address / Aktenzeichen), qr (decode GiroCode payment QR), elements (a layout GNN), and semantic (a cross-document evidence graph) — all keyless and self-contained.

selected command — prose return

    
Layers 1-2 structure: abstract, Markdown, facts, math and code-fences lifted from the page.
Layers 1–2 — structure: abstract, Markdown, facts, math and code-fences pulled from the page.

05 — the qc pipeline

LaTeX · KaTeX · the image MathPix actually rendered.

MathPix lines.json is the only format that ships, for each equation, both the recognized LaTeX and the CDN crop it was read from. pdfdrill puts them side by side — and adds competing readings (MathPix Snip, an LLM on the crop, the author’s own .tex) as extra columns, each scored against a normalized LaTeX form. Below is a real row from report --embed on arXiv 2004.05631v1, eq (1.1):

LaTeX sourceKaTeX renderMathPix CDN crop
a(x):=\{y \in Y \mid R(x, y)=1\} \quad b(y):=\{x \in X \mid R(x, y)=1\}
snip 1.00
tex  0.99
MathPix crop of eq (1.1)

Each crop’s <img> links to the full page it was cut from (page_url() strips the region query). --embed base64-inlines every crop so the artifact has no live-CDN dependency — ideal for a preview that won’t load remote images. The page click-through stays live even when embedded.

snip

pdfdrill snip re-OCRs each crop through MathPix Snip /v3/text → a candidate LaTeX + a per-line confidence that becomes a score signal.

vision / candidates

candidates exports the crops for an LLM to read (no API key); vision runs GPT-4o over them. Both attach as extra provenance columns.

tex (gold)

pdfdrill latex aligns the author’s e-print .tex to each MathPix equation — the gold reference versus OCR, preamble-expanded so TikZ/macros resolve.

Experimental · LaTeX render path

A test input path renders tables straight from LaTeX rather than KaTeX: latex → DVI → dvisvgm emits a font-faithful vector SVG that drops into the HTML table — useful for full tabular layouts KaTeX can’t express. It stays keyless: latex already ships in the Claude.ai sandbox and dvisvgm (3.2.1) is apt-installed by the bootstrap, so it runs there as readily as on a local machine.

Closed loop

Scoring flags shaky equations; escalate exports only those for a second reading; after ingest, relearn re-scores and reports resolved / improved / still-shaky. On 2605.12061: 9 flagged → 7 resolved, 1 retained (the hardest multi-line equation, correctly kept).

Layer 5 provenance: MathPix, Snip, vision and the author's gold .tex scored side by side as competing readings.
Layer 5 — drill · provenance: mathpix · snip · vision · the author’s gold .tex, each a scored column.

06 — live from the model

Real equations, rendered in your browser.

These are pulled verbatim from the extracted model of arXiv 2004.05631v1 (At the Interface of Algebra and Statistics) — the Galois connection at the heart of the thesis. Source LaTeX shown beneath each render; nothing was hand-typed.

Layer 7 QC report: each row LaTeX vs KaTeX render vs page image, with per-equation score chips and a flag.
Layer 7 — QC · report: LaTeX │ KaTeX │ image per equation, with score chips and review flags.

07 — the document model

Every element is a typed, queryable node.

The unified docmodel resolves a paper into typed DocObjects. Projected to TiddlyWiki, each becomes one tiddler — equations carry their LaTeX, reference number, page, the MathPix crop, region geometry, and every competing reading as latex_<provenance> + score_<provenance>.

Tiddler types in one extracted thesis

1116 formula (inline)
254 equation (display)
574 paragraph
137 diagram
135 page
134 sidenote
71 listitem
57 section
13 footnote
12 table
reference / bibentry
2618 tiddlers total

In-text (N) references rewrite to {{<eq>||FREF}} transclusions; citations link straight to their bibliographic tiddler by number or citekey. The persisted model.docmodel.json is the complex memory that survives between LLM/tool calls.

One equation tiddler, verbatim

{
  "title": "2004.05631v1_EQ0001_p012",
  "text":  "<$latex text={{!!latex}} displayMode=true />",
  "tags":  "equation 2004.05631v1",
  "kind":  "Equation",
  "latex": "a(x):=\{y \in Y \mid R(x,y)=1\} …",
  "refnum":          "1.1",
  "equation_number": "(1.1)",
  "page":   "012",
  "canonical_uri": "https://cdn.mathpix.com/cropped/…-012.jpg?…",
  "width": "994", "height": "67",
  "top_left_x": "259", "top_left_y": "666",
  "latex_tex":  "a(x):=\{y\in Y\mid R(x,y)=1\}…",
  "score_tex":  "0.992",
  "latex_snip": "a(x):=\{y \in Y \mid …\}",
  "score_snip": "1"
}
Beyond the tree — the semantic graph (CSP) preview

pdfdrill semantic lifts the model into a constraint graph: every extractor becomes a sensor emitting evidence, and entities — Company / Person / BankAccount on a letter, Symbol / Claim / Reference on a paper — accumulate it, across documents when you pass --store graph.json. The pass itself is reified: each reading is a Question node, each transform a first-class Transformation node, and a derived belief column carries a deterministic best-reading tiebreak that a compiler type-checks for grounding and consistency. A federated vocabnet layer (MSC · PhySH · GND · STW) backs pdfdrill classify, while stex / scikgtex project the graph back out to enriched LaTeX — glossary, Table-of-Symbols and index, or an ORKG-annotated PDF whose contribution metadata rides along as XMP/RDF.

Why TiddlyWiki

It’s a single self-contained HTML file — no server, perfect for GitHub Pages — that holds the whole graph and renders the LaTeX widgets live through the KaTeX plugin. The artifact below is exactly that: pdfdrill tiddlers output, built into a real wiki you can open and explore.

Layer 3 model: typed nodes with text and region streams, alignments between them, and region geometry, persisted as docmodel.json.
Layer 4 — the L3 model: typed nodes, text/region streams, alignments and region geometry → docmodel.json.

08 — claude.ai & claude code

Nothing for you to install — just ask Claude.

You don’t run any of this yourself. Make pdfdrill available once — attach the bundle in a Claude.ai chat, or install the skill in Claude Code — then ask in plain language. Claude carries the skill (SKILL.md + the 89-command manifest), so it knows which commands to run, and it reads that manual — not you. The package is pure Python under src/ with no pip step, which is why the sandbox (the extraction tools and pydantic already aboard) runs it as-is. Everything in the terminal below is what Claude does for you — not a checklist you type.

you ask in plain language — Claude loads the skill and runs pdfdrill for you
you ▸ “Here’s pdfdrill (attached). Install it and load its skill, then tell me
       whether this paper’s equations extracted cleanly.”

# Claude — now holding the skill — does the rest. You watch; you don’t type any of this:
$ tar xzf pdfdrill.tgz                        # unpack the bundle you attached
$ cat src/pdfdrill/skill/SKILL.md            # the skill IS Claude’s manual — it reads this, you don’t
$ PYTHONPATH=src python3 -m pdfdrill size   paper.pdf
$ PYTHONPATH=src python3 -m pdfdrill report paper.pdf --embed   # → self-contained formula-report.html

# In Claude Code the skill is installed once, so step one is simply your question.
# (the attachable bundle is one `git archive --format=tar.gz HEAD` of the repo.)

keys from the environment

mathpix, snip, bibfetch read MATHPIX_APP_ID/KEY & PERPLEXITY_API_KEY from env or a git-ignored .env — never committed, never asked for mid-task.

offline by default

The structural path — model, compare, report, tiddlers, folder, latex — runs entirely from an existing lines.json. No key, no network.

keyless OCR fallback

No MathPix? pdfdrill ocr renders pages → tesseract → a MathPix-compatible lines.json (--lang eng+equ for math, eng+deu for German), and model falls back to it automatically.

Keyless LLM via Claude preview · not fully tested

New: when there is no API key but pdfdrill is being driven by Claude — the Claude Code CLI locally, or Claude.ai in the sandbox — it routes the prompt-driven sub-tasks (vision, bibtex) to that Claude, using pdfdrill’s own prompts. The prompt is the knowledge: a vision crop returns as recompilable LaTeX, a BibTeX record web-searched correct — no OpenAI or Perplexity key required. pdfdrill llm <pdf> --runtime reports the transport it detected: cli (synchronous claude -p --output-format json) or sandbox (a deferred request file the agent answers, then re-run to ingest). The attached result is byte-identical to the keyed API path. But it is not free: delegating a whole document to the running Claude can take more wall-clock time and burn more tokens than a single keyed call — so for steady use, setting up MathPix pay-as-you-go is often cheaper and faster.

A first-class Claude skill new

pdfdrill now ships as a Claude skill. A single commands.yaml manifest is the source of truth for the whole 89-command surface — --help, the SKILL.md tables and the external drillui TUI are all generated from it, and a CI gate fails on any drift. A declarative prerequisite planner (pdfdrill steps, --ensure) chains the right commands toward a goal, so the agent asks for an outcome and pdfdrill walks the path.

Network access needed for the sandbox

In the settings menu, the Claude.ai sandbox can be set to allow network access to all hosts. Only after that change are the external services (MathPixmathpix, snip; ChatGPT / GPT-4ovision; Perplexitybibfetch; and Gemma), fetching a PDF by URL, and even apt-installing dvisvgm at bootstrap possible. The keyless offline path — model, compare, report, tiddlers, ocr — touches no host and needs no change.

Do not upload PDFs to Claude.ai

Attach a PDF in a Claude.ai chat and it is silently flattened to Markdown — math turns to Unicode and the annotation layer (every hyperlink with no visible anchor text) is discarded before pdfdrill ever sees it. So don't hand Claude the bare PDF. Do one of these instead: (1) share a URL to the PDF — pdfdrill fetches it byte-for-byte; or (2) put the PDF in a .zip and upload that — the archive sidesteps the auto-Markdown conversion, so the real bytes reach the tool.

If a PDF is already sitting in the sandbox, tell Claude to run pdfdrill on that file — not the converted text: pdfdrill model <uploaded.pdf>, then pdfdrill llmtext <uploaded.pdf>. The llmtext dump is written to the doc's drill folder as <name>.drill/<name>.llm.txt. pdfdrill does not feed that file into the chat on its own, so explicitly ask Claude to read the .llm.txt file from the drill folder before it answers.

Self-hosting? Runs on CoCalc.ai new

Prefer your own environment over the Claude.ai sandbox? A one-shot bash cocalc-setup.sh brings the CoCalc standard image up to spec — sudo apt-get for poppler-utils, dvisvgm and the LaTeX toolchain, then bun, uv/uvx and the Python deps into your user dirs. The drillui ask-the-document web terminal then runs there via bun run tools/drillui_bridge.ts, reached through CoCalc's port-forward. One CoCalc quirk: the terminal's WebSocket must carry the proxy path — paste wss://<host>/<project-id>/server/8787/ws into its Connect box. Full walkthrough: COCALC.md.

drillspace — try it in your browser new

Click, wait for the codespace, and the drillui terminal opens in your browser on a sample paper that is already drilled — nothing installed, no command to type. Ask it a question, or type a pdfdrill command name to run it on the open document.

The container is prebuilt, so the toolchain is already baked in rather than installed on launch. It needs a GitHub account and spends its free-tier quota.

⎇ open drillspace ↳ repository

09 — live artifacts

Open the real output.

All produced by pdfdrill and served straight from this repository — no build step, no server.

arXiv 2607.02234 · Purified OPSD: On-Policy Self-Distillation new · docmodel inspector

Transformer Circuits Thread · Verbalizable Representations Form a Global Workspace in Language Models new · distill reading view

arXiv 2004.05631v1 · At the Interface of Algebra and Statistics

bibkey heimUFT · Burkhard Heim — Unified Field Theory (study notes; derived model published with permission)

bibkey kolbe2018hubbard · Kolbe (2018) — Korrelation im Hubbard-Modell (German BA thesis, published with the author's permission)

Layer 6 projection: a projector turns each model node into one tiddler, assembled into a self-contained browser/HTML wiki.
Layer 6 — project: one model node → one tiddler → a single self-contained browser/HTML wiki.