Callimachus · Pipeline ↗
Parse; Resolve; Fetch; Verify; Report; Resume
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Citation verification tools compared
Compare how citation-verification systems document manuscript parsing, reference matching, evidence retrieval, claim assessment and reporting. Each entry links to its source and reviewed location. Sources
Prepared by the Callimachus maintainers from public materials reviewed 3 November 2026. Competitors were not executed, and no shared benchmark was run.
Comparison
“Not established” means the sources reviewed here did not document the mechanism. It does not establish that the feature is absent.
Symbols summarize the reviewed sources. A check means the sources document the behavior; it is not a runtime test. A cross means the item was excluded from the reviewed workflow or is not provided in the reviewed scope. A question mark means the sources reviewed did not establish the behavior. Partial coverage means the sources describe both a supported part and a specific limit. Missing information remains not established.
✓Documented in sources
≈Different scope or approach
◐Partial coverage
?Not established
✕Not provided in reviewed scope
| Workflow checkpoint | CallimachusCode and documentation | Bib | Deep | Ref | Semantic | citefactCode and documentation | CitatioProduct documentation | Cite | Ref | Sci | Reference | Verus |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Parse & attribute | ||||||||||||
| Manuscript inputWhich manuscript or per-citation inputs are documented? | ✓ DOCX, LaTeX, PDF, Markdown, text and HTML. DocumentedProject documentation Sources | ✓ LaTeX, XML/HTML and PDF. DocumentedResearch paper Sources | ≈ Claim and associated reference. Different scope or approachResearch paper Sources | ✓ Complete scientific manuscripts. DocumentedResearch paper Sources | ≈ Citation text plus supplied source text, file or URL. Different scope or approachCode and documentation Sources | ✓ Markdown manuscript; BibTeX or Zotero bibliography. DocumentedProject documentation Sources | ✓ PDF and DOCX, including Zotero-linked DOCX. DocumentedProduct documentation Sources | ✓ PDF manuscript and source PDFs. DocumentedProduct documentation Sources | ✓ Manuscript upload. DocumentedProduct documentation Sources | ≈ Text plus references, or explicit claim–source pairs. Different scope or approachProduct documentation Sources | ✓ Pasted text or uploaded DOCX, PDF and text. DocumentedProduct documentation Sources | ✓ PDF reference extraction using an LLM. DocumentedProduct documentation Sources |
| Citation mappingHow are claims or citation contexts linked to references? | ✓ Rule-governed citation links; ambiguous cases can require manual review. DocumentedProject documentation Sources | ✓ Markup anchors; PDF patterns with constrained LLM disambiguation. DocumentedResearch paper Sources | ≈ Pair provided; LLM parses reference fields. Different scope or approachResearch paper Sources | ✓ Extracts citation-bearing sentences and checks bibliography metadata. DocumentedResearch paper Sources | ≈ Explicit pair; LLM standardizes the citation into a claim. Different scope or approachCode inspected Sources | ✓ Deterministic author–year mapping; LLM claim extraction. DocumentedProject documentation Sources | ✓ References mapped to cited claims. DocumentedProduct documentation Sources | ✓ Citation links plus AI decomposition into individual claims. DocumentedProduct documentation Sources | ✓ Claims and references checked across the manuscript. DocumentedProduct documentation Sources | ✓ Auto-extracted citations or caller-provided pairs. DocumentedProduct documentation Sources | ✓ Bibliography checked before atomic claim comparison. DocumentedProduct documentation Sources | ✕ Inline-to-bibliography graph explicitly outside scope. Not provided in reviewed scopeProduct documentation Sources |
| Manual citation attribution reviewCan an operator adjudicate extracted citation/reference links before source resolution? | ✓ Optional hash-bound Parse tasks let an operator split sources, correct identity or retain ambiguity before Resolve. DocumentedProject documentation Sources | ≈ Manual inspection is documented for low-confidence evidence abstentions, not pre-resolution citation attribution. Different scope or approachResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ≈ Reviewer oversight is part of the citation-verification workflow; a pre-resolution Parse task is not established. Different scope or approachResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ✕ The pinned public run_check path parses and resolves citations without a manual pre-Resolve adjudication stage. Not provided in reviewed scopeCode inspected Sources | ≈ The described workflow sends uncertain verification cases to human review; pre-resolution citation attribution tasks are not described. Different scope or approachProduct documentation Sources | ≈ A human reviewer records claim-support verdicts after evidence review; a pre-resolution attribution task is not described. Different scope or approachProduct documentation Sources | ≈ The documented editorial workflow ends in an editor decision; pre-resolution attribution tasks are not described. Different scope or approachProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ≈ Users can upload source files and review results; pre-resolution citation-link overlays are not described. Different scope or approachProduct documentation Sources | ✕ Claim-bearing manuscript attribution review is outside the stated bibliography-verification scope. Not provided in reviewed scopeProduct documentation Sources |
| Identify the source | ||||||||||||
| Source familiesDoes the reviewed material describe source-type-specific handling beyond journal articles? | ✓ Scholarly, book, legal, clinical-trial and institutional-report routes. DocumentedProject documentation Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedProject documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ✓ A separate bibliography checker lists journals, proceedings, books, web sources, preprints, DOI records and other indexed categories. DocumentedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources |
| Bibliographic identityHow is a reference matched to a work? | ✓ Identifier and metadata corroboration; failed lookup is not proof of fabrication. DocumentedCode and documentation Sources | ✓ DOI/metadata resolution; OA-copy equivalence checks. DocumentedResearch paper Sources | ✓ Identifier/title cascade with a title-similarity gate. DocumentedResearch paper Sources | ✓ Bibliographic checks against scholarly databases. DocumentedResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ≈ Bibliography/source presence; metadata/DOI linting explicitly excluded. Different scope or approachProject documentation Sources | ✓ Reference resolution described; identity rules unspecified. DocumentedProduct documentation Sources | ✓ DOI, title, author and venue matching. DocumentedProduct documentation Sources | ✓ Metadata compared with Crossref, OpenAlex, PubMed and other databases. DocumentedProduct documentation Sources | ≈ DOI, URL and title accepted; validation rules not specified. Different scope or approachProduct documentation Sources | ✓ Reference verdict described as deterministic record comparison. DocumentedProduct documentation Sources | ✓ Crossref and web-search matching; metadata/hallucination flags. DocumentedProduct documentation Sources |
| Journal metadata checksWhich journal or venue identifiers and metadata checks are documented? | ✓ A versioned journal authority supports journal identity checks; resolver-coverage observations are recorded separately. DocumentedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ✕ Metadata and DOI linting are explicitly excluded from the reviewed citefact workflow. Not provided in reviewed scopeProject documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ✓ Reference resolution matches venue metadata, including journal information, to a real work. DocumentedProduct documentation Sources | ✓ Bibliographic integrity checks include journal and volume metadata against scholarly databases. DocumentedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ✓ A separate journal-check workflow reports indexing, quartile, discontinuation and publishing-risk signals. DocumentedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources |
| Journal registry provenanceDoes the run record which journal-registry version was used for its checks? | ✓ The run can record the version and hash of its local journal-authority registry. DocumentedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources |
| Issue contents checksDoes an official issue check compare the cited work with a publisher issue inventory or table of contents? | ◐ Official issue-inventory checks are implemented only for The George Washington International Law Review (GWILR). Partial coverageCode inspected Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources |
| Independent citation-style validationDoes the workflow check citation or bibliography formatting separately from identity and claim support? | ✓ Source-type-specific Vancouver, APA 7, Chicago and MLA 9 findings are independent of identity and semantic support. DocumentedProject documentation Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources |
| Acquire usable evidence | ||||||||||||
| Automatic acquisitionHow does the tool acquire source text? | ✓ Provider, OA, repository and corroborated-cache paths. DocumentedProject documentation Sources | ✓ Publisher APIs and OA repositories. DocumentedResearch paper Sources | ✓ URL, arXiv, OA APIs and LLM-assisted web-search fallback. DocumentedResearch paper Sources | ✓ Resolves references to OA PDFs. DocumentedResearch paper Sources | ≈ File/URL input; general bibliographic acquisition not established. Different scope or approachProject documentation Sources | ✕ No automatic DOI retrieval; user supplies sources. Not provided in reviewed scopeProject documentation Sources | ✓ Available sources, prioritizing OA full text. DocumentedProduct documentation Sources | ✓ OA PDFs retrieved when available. DocumentedProduct documentation Sources | ≈ Source comparison stated; acquisition mechanics not specified. Different scope or approachProduct documentation Sources | ≈ Full-text retrieval described for the follow-on ask-paper route. Different scope or approachProduct documentation Sources | ✓ Accessible full text; abstract otherwise. DocumentedProduct documentation Sources | ? Metadata lookup is described; acquisition of cited source text for claim-support assessment is not established. Not establishedProduct documentation Sources |
| Assisted retrievalWhat operator-supplied sources, recovery or OCR paths are described? | ✓ Guided Fetch, user files, browser challenges, OCR and resume. DocumentedProject documentation Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ✓ User-supplied reference document. DocumentedProject documentation Sources | ✓ Zotero/local PDFs; Docling-based OCR. DocumentedProject documentation Sources | ✓ Users can provide reference PDFs; the public workflow also describes open-access and publisher-access retrieval. DocumentedProduct documentation Sources | ✓ Source uploads and correction of extracted records. DocumentedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ✓ Explicit pairs can include full_text. DocumentedProduct documentation Sources | ✓ User files; DOI/title-author matching; ambiguous matches require user choice. DocumentedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources |
| Evidence boundariesHow are unavailable sources and partial evidence treated? | ✓ Full text, abstract and preview scopes remain distinct; generic web evidence excluded. DocumentedProject documentation Sources | ≈ Accessible versus inaccessible routes; downstream-citer consensus under paywalls. Different scope or approachResearch paper Sources | ≈ Abstract verdict can end the run; NEI triggers full-text retrieval. Different scope or approachResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ✓ Uncertain is an explicit assessment class. DocumentedResearch paper Sources | ✓ Truncation flagged; failed model calls remain unverified. DocumentedProject documentation Sources | ✓ Unavailable citations remain visible. DocumentedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ≈ Full-text/abstract search modes documented for /verify, not all citation checks. Different scope or approachProduct documentation Sources | ✓ Inaccessibility is distinguished from claims not found in supplied text. DocumentedProduct documentation Sources | ✓ Unmatched entries flagged for investigation. DocumentedProduct documentation Sources |
| Assess the relationship | ||||||||||||
| Semantic assessmentWhat component evaluates the claim–source relationship? | ✓ The model assesses each claim against its linked reference; deterministic code decides which outcomes are accepted. DocumentedProject documentation Sources | ✓ Dense retrieval, cross-encoder, NLI, then reasoning-model escalation. DocumentedResearch paper Sources | ✓ Abstract-level or passage-level LLM judgment. DocumentedResearch paper Sources | ✓ Verdicts with natural-language explanations. DocumentedResearch paper Sources | ✓ Four-class LLM assessment. DocumentedResearch paper Sources | ✓ Per-pair LLM suggestion with evidence and reasoning. DocumentedProject documentation Sources | ✓ AI citation validation with graded support findings. DocumentedProduct documentation Sources | ≈ Human records Supported / Partial / Unsupported. Different scope or approachProduct documentation Sources | ✓ Claim fidelity checked separately from bibliographic integrity. DocumentedProduct documentation Sources | ✓ Support, overstatement, contradiction and non-addressed claims. DocumentedProduct documentation Sources | ✓ Meaning-level AI comparison on the vendor’s server. DocumentedProduct documentation Sources | ✕ Claim-support assessment explicitly outside scope. Not provided in reviewed scopeProduct documentation Sources |
| Context selectionWhat source context is selected for the assessment? | ✓ Complete context or recorded extractive BM25 source ranges. DocumentedCode and documentation Sources | ✓ Ranked paragraphs and sentence windows. DocumentedResearch paper Sources | ✓ Dense passage retrieval; LLM extraction tested as an ablation. DocumentedResearch paper Sources | ✓ Evidence-passage localization. DocumentedResearch paper Sources | ✓ BM25 + dense retrieval + FlashRank. DocumentedCode inspected Sources | ≈ Source-character cap, with explicit truncation flags. Different scope or approachProject documentation Sources | ✓ Citation-relevant excerpts. DocumentedProduct documentation Sources | ✓ AI locates passages with source-document locations. DocumentedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ✓ Citation comparison uses accessible full text or an abstract; users may upload closed-source full text with file/page attribution. DocumentedProduct documentation Sources | ✕ Context selection for claim-support assessment is outside the stated workflow because claim support is explicitly excluded. Not provided in reviewed scopeProduct documentation Sources |
| Second semantic reviewCan a second automatic pass check whether the cited passages justify the proposed support finding? | ✓ Policy-controlled Jury2 can check whether the cited passages justify the proposed support finding. DocumentedProject documentation Sources | ≈ ACSV escalation and ICSV committee consensus are documented multi-stage routes; a separate pass over a proposed support finding is not established. Different scope or approachResearch paper Sources | ≈ Abstract-first verification escalates inconclusive cases to passage-level analysis; a separate second pass over the proposed finding is not established. Different scope or approachResearch paper Sources | ? Reviewer oversight is documented, but the public abstract and reproduction package do not establish whether a separate automatic semantic pass exists. Not establishedResearch paper Sources | ✕ The pinned public per-citation workflow assigns a finding without a second semantic review. Not provided in reviewed scopeCode inspected Sources | ✕ The pinned public workflow assigns each claim a verdict without a second semantic review. Not provided in reviewed scopeCode inspected Sources | ≈ An AI first pass and human escalation for uncertain cases are documented; a separate automatic second pass is not established. Different scope or approachProduct documentation Sources | ≈ A human reviewer records the support verdict; a separate automatic second pass is not established. Different scope or approachProduct documentation Sources | ≈ Claim-fidelity and bibliographic checks are documented as separate tasks, followed by an editor decision; a second semantic pass is not established. Different scope or approachProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ≈ The documented workflow checks bibliography metadata and then compares claim meaning; a separate automatic review of the same finding is not established. Different scope or approachProduct documentation Sources | ✕ Claim-support review is explicitly outside the stated workflow, so a second semantic pass is out of scope. Not provided in reviewed scopeProduct documentation Sources |
| Mechanical groundingIs checking returned evidence against admitted source text documented? | ✓ Selected passages are checked against admitted source text, recorded offsets and the source hash. DocumentedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ≈ Manuscript quotations checked mechanically; model-evidence validation not established. Different scope or approachProject documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ≈ Located source quotations stated; internal validation contract not published here. Different scope or approachProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ✕ Mechanical grounding of claim-support evidence is outside the stated workflow because claim support is explicitly excluded. Not provided in reviewed scopeProduct documentation Sources |
| Keep the record | ||||||||||||
| Findings and source context retainedWhat findings and source context are retained for later review or reuse? | ✓ Typed findings, evidence scopes, source hashes, attempts and provenance remain in the run record for later review. DocumentedProject documentation Sources | ≈ Per-verdict witness bundles, evidence statements and votes are described, but retention of the complete cited-source assessment context is not established. Different scope or approachResearch paper Sources | ≈ Stage logs and an in-memory cache are described; durable retention of the full assessed source context is not established. Different scope or approachResearch paper Sources | ≈ Per-item evaluation outputs and run manifests are published; retained user-run assessment context is not established. Different scope or approachResearch paper Sources | ≈ Per-verdict snippets and Markdown export are documented; retention of the full assessed source context is not established. Different scope or approachCode and documentation Sources | ✓ Conversion, extraction and verdict caches are content-addressed; HTML/JSON findings include model evidence and source location. DocumentedCode and documentation Sources | ? Reports include evidence excerpts, and the vendor describes a 30-day deletion period; complete assessed source-context retention is not established. Not establishedProduct documentation Sources | ✓ Private account files and the portable .cfv record retain claims, source passages and reviewer verdicts. DocumentedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ◐ Saved reports retain findings and evidence; reruns require the source files to be uploaded again. Partial coverageProduct documentation Sources | ≈ CSV and bibliography exports retain metadata results; retention of cited-source assessment context is not established. Different scope or approachProduct documentation Sources |
| Resume interrupted workCan an interrupted run continue from saved phase or task state? | ✓ Typed SQLite run state and task provenance support resuming interrupted pipeline work. DocumentedProject documentation Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ✕ The pinned public workflow can reuse content-addressed caches on a rerun but has no persisted phase/task state for continuing an interrupted run. Not provided in reviewed scopeCode inspected Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources |
| Report outputWhich reports, result records or exports does the reviewed workflow produce? | ✓ Deterministic Markdown and HTML reports present run health, findings, source evidence and provenance. DocumentedProject documentation Sources | ✓ Per-citation findings, error taxonomy and traceable evidence. DocumentedResearch paper Sources | ✓ Final support / contradiction / NEI label. DocumentedResearch paper Sources | ✓ Reviewer-facing evidence and explanations. DocumentedResearch paper Sources | ✓ Reasoning, evidence snippets and Markdown export. DocumentedProject documentation Sources | ✓ Self-contained HTML; versioned JSON for CI. DocumentedProject documentation Sources | ✓ Interactive audit with evidence excerpts. DocumentedProduct documentation Sources | ✓ Shareable claim, source, passage and human-verdict record. DocumentedProduct documentation Sources | ✓ Editorial report includes claim issues and source evidence. DocumentedProduct documentation Sources | ✓ Evidence cards for checked citations. DocumentedProduct documentation Sources | ✓ Evidence, uploaded-file/page attribution and Word/CSV export. DocumentedProduct documentation Sources | ✓ Editable result table; CSV and bibliography exports. DocumentedProduct documentation Sources |
| Report reconstructionCan a report be regenerated from retained execution data without repeating claim assessment? | ✓ The canonical report and HTML companion are deterministic projections of current typed run data. DocumentedProject documentation Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources |
| Report integrity checksWhat checks bind report content to run data or detect later changes? | ✓ The report gate checks its deterministic projection and hash-linked journal; the HTML seal binds the report, projection, assets and body. DocumentedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources |
| Models & deploymentCan operators select models or run inference in their own environment? | ✓ Operator-selected local / hosted backends; public AGPL-3.0-only code. DocumentedProject documentation Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ≈ API models in reported setup; local deployment not established. Different scope or approachResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ✓ Cloud providers or local Ollama; specialized Qwen models described. DocumentedProject documentation Sources | ✓ LiteLLM providers, including local Ollama. DocumentedProject documentation Sources | ≈ Web application with AI inference providers. Different scope or approachProduct documentation Sources | ≈ Hosted accounts with third-party AI processing. Different scope or approachProduct documentation Sources | ≈ Standalone platform or editorial-system API integration. Different scope or approachProduct documentation Sources | ≈ Hosted REST API with developer credentials. Different scope or approachProduct documentation Sources | ≈ Model runs on vendor infrastructure, not an established customer-local route. Different scope or approachProduct documentation Sources | ≈ Hosted upload-and-review workflow. Different scope or approachProduct documentation Sources |
| Human judgmentWho assigns the suggested result, and what role remains for the reader? | ✓ Recorded AI findings inform review; grounding does not prove semantic truth. DocumentedProject documentation Sources | ✓ Low-confidence ACSV abstentions surface evidence windows for manual inspection. DocumentedResearch paper Sources | ? Not established in the reviewed sources. Not establishedResearch paper Sources | ✓ Semi-automated assessment with reviewer oversight. DocumentedResearch paper Sources | ? Not established in the reviewed sources. Not establishedCode and documentation Sources | ✓ AI suggestions explicitly not ground truth. DocumentedProject documentation Sources | ✓ Final interpretation remains with authors, reviewers and editors. DocumentedProduct documentation Sources | ✓ Human verdict required; research aid, not certification. DocumentedProduct documentation Sources | ✓ Editor decides whether to proceed or request corrections. DocumentedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ✓ Results described as decision support, not a verdict. DocumentedProduct documentation Sources | ✓ Users inspect findings and can change statuses. DocumentedProduct documentation Sources |
| Evaluation materialWhat public evaluation material was found in the reviewed sources? This is not a shared leaderboard. | ✓ Example report demonstrates the interface, not independently attested accuracy. DocumentedProject documentation Sources | ✓ Paper reports MisciteBench experiments. DocumentedResearch paper Sources | ✓ SCitance experiments; code/data on request. DocumentedResearch paper Sources | ✓ Stage evaluations and eight-manuscript end-to-end study; reproduction outputs. DocumentedResearch paper Sources | ✓ Paper reports model evaluation and a citation dataset. DocumentedResearch paper Sources | ≈ Mocked unit tests and opt-in LLM evaluation fixtures. Different scope or approachProject documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources | ? Not established in the reviewed sources. Not establishedProduct documentation Sources |
How to read the entries
The matrix summarizes the responsibilities and interfaces described in the cited public material. Each Sources link gives the evidence basis, reviewed location and any recorded qualification.
Research papers, project documentation, selected source code and vendor pages describe different scopes. The table records those differences without combining them into a measured accuracy or performance result.
Unknown cells
“Not established” records a limit in the sources reviewed. Proprietary internals and unreviewed implementations are not assumed absent.
Versions and coverage
Callimachus entries refer to a pinned source snapshot. Listed providers and source families describe supported paths. Access to a document depends on the source and configuration. Sources
Evidence and scientific judgment
Mechanical grounding checks returned passages against admitted source text. It does not determine whether a model's interpretation is scientifically correct. Callimachus can run a second assessment of the selected passages, called Jury2. Its use and enforcement depend on the configured policy. Sources
Evidence sources
Links, reviewed locations, dates and available version details. Reviewed 3 November 2026.
Callimachus14 records
Callimachus · Verification and evidence ↗
Evidence scopes; identity admission; Jury contracts; deterministic grounding; policy
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Callimachus · Artifacts and provenance ↗
SQLite projections; source-file lifecycle; provenance chain; report artifacts
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Callimachus · Capabilities and limits ↗
Manuscript formats; source families; Resolve/Fetch; explicit limits
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Callimachus · Journal authority ↗
Versioned journal registry; ISSN normalization/checksum; hashed registry snapshot
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Callimachus · Bibliographic adjudication ↗
Deterministic adjudication; independent DOI/PMID corroboration
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Callimachus · Scoped publisher issue attestation ↗
Official issue-TOC adapter; explicit applicability boundary
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195This particular adapter is scoped to The George Washington International Law Review, not all journals.
Callimachus · Desktop, Guided Fetch and model selection ↗
Review sources and continue work; choose models; example-report limitations
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Callimachus · Public licence ↗
AGPL-3.0-only; component exceptions; alternative written licensing
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Callimachus · Grounding implementation ↗
ground_jury1_decision; ground_jury1_span_decision; raw-range validation
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Callimachus · Context retrieval ↗
select_bm25_ranges; pinned BM25; raw source-span ranges
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Callimachus · Backend configuration ↗
Journal authority and resolver coverage; registered local and hosted backends
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Callimachus - Human report seal ↗
HTML seal creation and verification against report, live projection, assets and renderer
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Seal algorithm depends on trusted signing-key availability.
Callimachus - Integrity signing helper ↗
HMAC-SHA256 with trusted key; SHA-256 content binding without key
Version / snapshot: 45b848d5516c599215c066ba02892efa8ca96195Do not describe local degraded mode as HMAC-signed.
Research systems14 records
BibAgent · Paper and technical appendices ↗
§3.1–3.3; Appendix C.1–C.5; D.1; D.7
Version / snapshot: arXiv:2601.16993v2Described architecture; its application implementation was not audited for this page.
DeepSciVerify · Paper and implementation details ↗
§3–4; Appendix B; Appendix C
Version / snapshot: arXiv:2605.27710v1Paper-based comparison. The paper says code and data are available on request.
RefVerifier · Official conference abstract ↗
Abstract: workflow, evidence localization and reviewer oversight
Version / snapshot: ATIQSER 2026RefVerifier · Reproduction package ↗
Layout; ground truth; runs; human-subjects data; licence
This reviewed package describes evaluation inputs and outputs; it is not evidence of a complete released application implementation.
SemanticCite · Public implementation ↗
Reference preparation; LLM claim processing; BM25, dense retrieval and FlashRank
Version / snapshot: 813848c1026df65f4cbb1f21958cdc726d62172eSemanticCite · Public interface and deployment ↗
Python API; CLI; local dual-model configuration; tailored full-document service
Version / snapshot: 813848c1026df65f4cbb1f21958cdc726d62172eMatrix refers to the public per-citation implementation, not uninspected enterprise offerings.
SemanticCite · Research paper ↗
Abstract: four-class assessment; model evaluation; dataset
Version / snapshot: arXiv:2511.16198v1citefact · Workflow and explicit limitations ↗
Three checks; How It Works; Limitations; LLM configuration; developer tests
README reviewed; implementation internals were not independently audited.
citefact - Public run pipeline ↗
Pinned public run_check flow from input loading through report construction
Version / snapshot: 469bb993e4047b5f3679877bdd4c710485e4e316Scoped to this public application path at this commit.
citefact - Verdict request and schema handling ↗
suggest_verdict request, JSON schema parsing and verdict normalization
Version / snapshot: 469bb993e4047b5f3679877bdd4c710485e4e316The claim-extraction pass is a separate function outside this line range.
citefact - Claim verdict orchestration ↗
check_claims extraction, per-claim verdict call/cache and report finding construction
Version / snapshot: 469bb993e4047b5f3679877bdd4c710485e4e316Confirms one semantic verdict call or cached result per claim in the scoped public path.
citefact - JSON response formatting repair ↗
One-shot malformed-JSON repair retry
Version / snapshot: 469bb993e4047b5f3679877bdd4c710485e4e316The retry requests JSON formatting repair, not semantic reassessment.
SemanticCite - Public per-citation verdict path ↗
One _analyze_support classification in check_citation after preprocessing and retrieval
Version / snapshot: 813848c1026df65f4cbb1f21958cdc726d62172eScoped to the pinned public per-citation implementation; not enterprise offerings.
SemanticCite - Public app call site ↗
Streamlit app calls check_citation for the submitted citation/reference pair
Version / snapshot: 813848c1026df65f4cbb1f21958cdc726d62172eConfirms the scoped public app uses the inspected path.
Public products12 records
Citatio · Product workflow ↗
Upload; Retrieve evidence; Review audit; human review; privacy-oriented processing
Public product description, not an independent implementation or performance test.
CiteFirm · How it works ↗
Extraction; resolution; source attachment; claim decomposition; human verdict; export
Version / snapshot: Page updated 2026-07-29CiteFirm · FAQ ↗
Verdicts; accepted formats; .cfv export; third-party AI processing
Version / snapshot: Page updated 2026-07-29WriteWise RefCheck · Product description ↗
Citation fidelity; bibliographic integrity; editorial workflow; API integration
Workflow claims are vendor descriptions. No inference is made about unpublished internals.
WriteWise RefCheck · Editorial workflow detail ↗
Bibliographic checking against Crossref, OpenAlex, PubMed; source evidence; editorial decision
SciTrue · Citation-checking API ↗
POST /v1/check-citations; explicit pairs; source inputs; evidence cards
The citation-check endpoint is compared here, not a claim of equivalence with all SciTrue operations.
ReferenceVerify · Usage guide ↗
§4 Verify citations; source-file matching; full text/abstract; report; deployment
ReferenceVerify · How to verify citations ↗
Bibliography verification; atomic claims; decision support; inaccessible sources
ReferenceVerify · Reference-verdict method ↗
Deterministic reference verdict against scholarly records
VerusCite · Explicit scope ↗
What VerusCite is; What VerusCite is not
Version / snapshot: Page dated 2026-06-20VerusCite · Product workflow ↗
LLM extraction; Crossref and web search; review and export
Citatio - About and workflow ↗
Reference PDFs; AI first pass and human escalation; 30-day deletion
Public product description; not independent implementation or evaluation.