Callimachus is an open-source tool that verifies the citations in a manuscript: whether each source exists and is the one cited, is formatted correctly, and actually supports the claim. It uses AI for the judgement calls — but confines it inside a deterministic pipeline, so the model proposes and the system controls the record.
$ python run.py --input manuscript.pdf --accuracy standard
Sample findingAttention Is All You Need
The claim in the manuscript
“RNN sequence-to-sequence models have not been able to attain state-of-the-art results in small-data regimes[37].”
↓ cites [37] Vinyals et al., 2015 — fetched and read in full
What that source actually says
“… when training on [the small training set] only, a baseline LSTM does not achieve any reasonable score, even with dropout and early stopping.”
The verdict
Partially supported
The source shows one model struggling on limited data — not that these models can’t. Its own attention variant scores competitively, so the broad claim isn’t established.
Most checkers answer one question — does the reference exist, or is it formatted right — and hand you a number. Callimachus refuses to collapse identity, form, and support into a single score. A reference can be real but misidentified, correctly formatted but irrelevant, or on-topic yet unsupported by the text. Those stay separate — and every answer keeps its evidence.
01
Identity
Does the source exist, and is it the cited source?Resolved before any semantic support is credited.
02
Citation form
Does the bibliography entry pass the deterministic checks for its style?Style findings stay independent of identity and support.
03
Claim support
What does this source establish about this exact claim?Supported, partial, contradiction, related, off-topic, or non-decidable.
[ 02 ] How it works
The model proposes. The system controls the record.
Callimachus uses a language model where judgement helps — weighing one claim against one source — but keeps it on a short leash. The model may classify a bounded claim–source pair and select passages. It cannot admit a source, bypass grounding, write a terminal state, or compose the report. A deterministic Python driver owns every phase transition, and the run fails closed when required proof or integrity is missing.
1.0
Parse
Read the manuscript — DOCX, LaTeX, PDF, Markdown, plain text, or HTML — into claims and references.
Deterministic
2.0
Resolve
Establish whether each source exists and is the one actually cited, before any support is credited.
Deterministic
3.0
Fetch
Retrieve and normalise the source text; identity and content hashes are retained as evidence.
Deterministic
4.0
Verify
A guarded jury judges one claim–source pair at a time. A proposal that fails the guard credits nothing.
Guarded jury
5.0
Report
Compose the deterministic report and audit ledger from admitted results only.
Deterministic
parse → resolve → fetch → verify → report
example · C7 / jury1_guard
[ 03 ] No silent recovery
When no admissible result survives, the report says so.
Here the Jury 1 proposal failed the verification guard. The pair stayed operationally complete but non-crediting — no plausible replacement verdict was invented to fill the gap. Missing evidence is recorded as missing, not converted into success.
Semantic outcomeNone
Resolutionjury1_guard
Creditingfalse
Evidence scopefulltext_complete
[ 04 ] What you get
The result is a record, not a paragraph.
Read the report first, then follow any finding down to the retained evidence, the source identity, the model's attempt, and the deterministic finalisation that produced it. The run ledger is an authoritative, auditable record — with a report history and integrity seal.
report.htmlSelf-contained, reader-first companion
report.mdDeterministic final report
run.sqliteAuthoritative audit ledger
sources/Normalised evidence with identity and hashes
journal + sealReport history and signature status
On the name & the mark
Callimachus of Cyrene compiled the Pínakes at the Library of Alexandria — the catalogue that first answered which scroll was truly which. This tool asks the same of every source a manuscript cites, and keeps the evidence attached to the answer.
The mark writes it as [C]: at once a citation reference and a harness — the brackets that let the model weigh a claim, but never let it write the record.
[ C ] Inspect before you trust
Citation verification with the evidence still attached.
