document-classification-v2 · commercial
Classify any document image against labels you choose at runtime. document-classification-v2 is an
open-vocabulary, zero-shot document classifier: supply one label + a short description per class at
inference, and it scores a document page against any such set — invoices, letters, memos, forms,
scientific articles, resumes, and whatever label you type next. No fixed class list, no per-class training.
It is the document specialist in a field of generalists. General open-vocab models were trained on web
photos; this one is built for document pages — and it shows on the
leaderboard. Weights are
commercial (not downloadable here); this page is a spec + scorecard.
Results
Macro-F1, zero-shot, on the held-out benchmark — higher is better. The flagship vs the previously-shipped
in-house model and a cloud frontier VLM reference.
| Benchmark (macro-F1) |
document-classification-v2 |
best generalist |
best cloud VLM |
| DocLayNet (page types) |
0.97 |
0.63 |
0.83 |
| Forms |
1.00 |
0.11 |
1.00 |
| Tobacco (doc types) |
0.74 |
0.43 |
0.85 |
| OOD (unseen doc types) |
0.95 |
— |
— |
| OOV (synonym wording) |
0.83 |
— |
— |
Latency (fused image+text, A40): ~4.9 pages/s on text-heavy pages; the image branch alone runs ~8 docs/s
when no OCR is present. The open-weight
v1 runs ~5.7 pages/s fused on the same GPU.
Every model — ours and cloud — is scored by the same open macro-F1 scorer; full per-model ranking on the
leaderboard.
- Matches/leads the cloud on visual document-type tracks (DocLayNet, Forms) at zero per-request API
cost — and runs on a single GPU.
- Trails on Tobacco (0.74 vs 0.85): that track rewards reading fine header text (memo vs letter vs
email) — a large VLM reads it; an embedding model can't.
- OOD robustness: on document types absent from training (invoices, handwriting, charts, tables), 0.95
macro-F1. Open-vocab (OOV): under never-seen synonym label wording, 0.83 — matches the concept, not
your exact string. (Cloud VLMs train on ~all data, so these held-out axes aren't reported for them.)
Intended use & limits
- Use it for: zero-shot classification of document page images in a free-label setting — the caller
supplies the candidate labels (and optional descriptions). Multi-page documents supported; optional page
OCR sharpens fine-grained form / tax-code distinctions.
- Limits: optimized for document imagery; English label strings are the primary target. Scores are
per-label match probabilities (independent per label), not a softmax across the set.
License & data
The model weights are offered under a commercial Nutrient license — deployed on-prem, so your documents
never leave your infrastructure. The training set is not redistributed. Evaluation runs on the held-out
document-classification-benchmark.
📩 Get access
document-classification-v2 is commercial and its weights are not downloadable here. To run it on-prem —
open-vocabulary, calibrated, private — contact Nutrient:
nutrient.io/contact-sales.
About the author
This project is maintained and funded by Nutrient - The deterministic document infrastructure enterprises run their highest-stakes workflows on: replayable output, clear exceptions, and full audit trails on the messy, regulated documents where AI alone breaks.