OCR Benchmark 2026: Unlimited-OCR vs PaddleOCR, Docling, LlamaParse and Surya

Updated October 16, 2026. We evaluated Baidu's Unlimited-OCR on October 14. It led the LLM judge scores in all four evaluation sets. The study now contains 1,999 graded results from 300 source documents, with the same invoices graded twice.

NEWTUPLE

OCR BENCHMARK · DOCUMENT-TYPE EDITION · JUNE RESULTS + OCTOBER UPDATE

In June, we expected one OCR engine to win. 1,600 evaluations on Gaugetuple proved otherwise.

PaddleOCR, Docling, LlamaParse and Surya, graded against fixed ground truth across invoices, financial tables and mixed real-world documents. The strongest engine changed with the document every time, which is the finding most teams standardising on a single OCR have not priced in.

01 · INTRODUCTION · JUNE BASELINES

Why we benchmarked four OCR engines

Optical character recognition is the first step in almost every document-AI system. Before a model can answer a question about an invoice or summarise a financial filing, some engine has to turn the page into text, and the quality of that text sets a ceiling on everything that follows. Most teams choose an OCR engine the way they choose any component: they read a public benchmark, pick whatever sits at the top, and assume it will hold up across the documents they actually process. We wanted to test that assumption rather than inherit it. This builds on our previous evaluation of finance-friendly OCR tools.

To do that, we ran four widely used engines over the same body of documents and graded every output the same way. The engines were PaddleOCR, Docling, LlamaParse and Surya. The documents covered three families that stress an engine in different ways: clean invoices with predictable layouts, dense financial tables with multi-column totals, and a mixed real-world set drawn from the OHR-Bench corpus. Our selected 100 pages are financial-report material. Each output was compared against a fixed ground-truth answer on Gaugetuple, our evaluation platform, which returned a score from zero to one, a categorical verdict, and a tag describing the kind of error when one occurred. The full run produced 1,600 graded outputs.

For the June engines, the finding that shaped the rest of this report is that no single engine was strongest across the board. Docling led comfortably on structured documents and then dropped to third on the mixed set. LlamaParse, which trailed Docling on invoices, won the hardest category outright. PaddleOCR rarely finished first in any category and rarely failed badly in one either, which turned out to matter more than any single peak score. The sections that follow work through the results by document type, by failure mode, and by the operational cost of choosing the wrong engine for a given workload.

Results at a glance

The sections marked June below describe the original four engines. Unlimited-OCR was added in October; its results and deployment trade-offs follow here.

The practical takeaway is that there are different OCR libraries that are suitable for different use cases.

Among the June engines, structured documents favoured Docling, mixed pages favoured LlamaParse or PaddleOCR, and PaddleOCR made a consistent default. The October addition changes the primary-engine shortlist; the June routing remains useful as a fallback.

UPDATE · OCTOBER 2026

Four months later, a new engine led every evaluation set

The June version of this report ended with the advice to re-benchmark. Baidu released Unlimited-OCR, an open-source vision-language OCR model with roughly 3 billion parameters and an MIT license, on June 22. On October 14 we evaluated its outputs against the same reference answers on Gaugetuple.

It led the mean LLM judge score in all four evaluation sets: narrowly on invoices, and by a larger margin on financial tables and mixed financial-report pages. The four June engines were not rerun with current releases, so this compares October Unlimited-OCR with June baselines.

Mean judge scores by engine and evaluation set, including Unlimited-OCR

Unlimited-OCR (orange) against the four June baselines. Scores range from zero to one; they are not character-accuracy percentages. Invoice A and B reuse the same 100 source invoices.

Leads are computed from unrounded means. There are 100 scores per engine and set except Unlimited-OCR Invoice A, with 99 completed scores. Small invoice gaps are not evidence of a statistically established advantage.

Financial tables were the clearest case for a trial. Unlimited-OCR scored 0.927, and 87 of 100 outputs were graded good or excellent. Most of its tagged table problems involved alignment: 22 outputs received a table-alignment tag, six missing-values tags and one a field-mapping tag.

Mean judge score across the four evaluation sets for five engines

Unlimited-OCR's highest-to-lowest mean-score gap was about 0.082, compared with PaddleOCR's June gap of about 0.164. These are differences across samples, rather than a controlled test of scan damage or increasing difficulty.

On mixed pages, Unlimited-OCR left six of 100 outputs below the good-or-excellent bar. Scaled to 1,000 outputs, that is 60 review candidates, against 320 for PaddleOCR, 330 for LlamaParse and 620 for Docling. On financial tables the corresponding comparison is 130 for Unlimited-OCR and 210 for PaddleOCR.

Estimated review candidates per 1,000 outputs on tables and mixed pages; Unlimited-OCR has 130 and 60 respectively

Review-load proxy = 1,000 × (1 − the good-or-excellent share). Each underlying rate is based on 100 outputs. Actual review hours, costs and production savings were not measured. Surya is omitted from this chart; its corresponding values are 930 and 980.

Two trade-offs matter before switching. On mixed pages, the judge tagged 73 outputs as hallucination, often citing headers, page numbers or other material absent from the reference. Those tagged outputs averaged about 0.936. A reference mismatch alone does not prove text was invented: inspect the source page before removing useful content.

The evaluation ran one page per call on an NVIDIA L4 in Gundam mode. Its timing summary reports mean times of 21.2 seconds per invoice, 10.3 seconds per financial table and 29.3 seconds per mixed page. This was not a hardware-matched speed comparison with the other engines.

Different metrics give different rankings. A separate recall analysis in the evaluation package puts Docling first on invoice field recall (100.0%, against Unlimited-OCR's 99.8%) and PaddleOCR first on OHR-Bench word recall (91.9%, against 81.6%). The package notes that this word-recall calculation does not strip HTML tags, creating a formatting confound for Unlimited-OCR's tables. These figures are separate from the LLM judge scores and should not be merged into one accuracy leaderboard.

The October results change our shortlist for accuracy-sensitive intake, especially tables and mixed financial-report pages. They do not establish the best engine for every metric, product version or workload.

02 · SCORE BY DOCUMENT TYPE · JUNE BASELINES

The ranking changes with the document type

The clearest way to see the result is to hold the engine constant and watch how its score moves across document types. When we do that, the order on the leaderboard does not survive the move from one document family to the next. An engine that reads invoices almost perfectly can sit in third place on a mixed corpus, and the engine that wins the hardest set is not the one that wins the easiest.

June mean scores by engine and document type

On invoices the four engines score within a few points of one another. On financial tables and on the mixed set the spread widens to more than thirty points, and the engine in first place is no longer the one that led on invoices.

Industry mapping is approximate. FinTabNet supplies financial-statement tables, the invoices are accounts-payable material, and our OHR-Bench subset contains financial-report pages.

03 · ROBUSTNESS · JUNE BASELINES

PaddleOCR holds up best as documents get more complex

Peak accuracy on a favourable document type tells only part of the story. For a production pipeline that sees a range of inputs, what matters as much is how far an engine falls when the documents get harder. Ordering the four test sets from the easiest invoices to the most demanding mixed pages lets us read each engine as a trajectory rather than a single number, and the trajectories separate the specialists from the all-rounders.

June scores across the evaluation sets

PaddleOCR has the flattest line and the smallest fall, losing 0.16 from its easiest to its hardest set. Docling starts highest of all but gives back the most ground among the leaders, a drop of 0.30. Surya starts low and declines by 0.45, settling at the bottom of every set.

June good-or-excellent rates by engine and set

Pass rate counts excellent or good outputs. The story sits in the right two columns: on financial tables and mixed documents, PaddleOCR holds 79% and 68% while Docling falls to 69% and 38%.

04 · HOW FAILURES CHANGE BY DOCUMENT TYPE · JUNE BASELINES

Each document type fails in its own way

A lower average score does not explain itself. Two engines can land on the same number for very different reasons, and the kind of mistake an engine makes matters more for a downstream pipeline than the size of the score. When we group the tagged failures by document type, the documents are not simply harder or easier than one another. They fail differently, and the dominant error class shifts as the layout changes.

June quality-verdict distribution

Verdicts for Invoice Run B are derived from score bands, because that run shipped without verdict labels. The other three sets carry the grader's own verdicts.

June primary error tags by document type

On structured documents the failures are overwhelmingly broken table alignment, where the text is read correctly but lands in the wrong cell. The mixed set looks different rather than simply worse: it introduces a large share of hallucinations and numeric errors, the two classes that are hardest to catch once the output moves downstream.

05 · CAPABILITY PROFILE · JUNE BASELINES

Strengths and weaknesses, engine by engine

June engine capability comparison

The radar plots each engine across five dimensions, all normalised to a zero-to-one scale: accuracy on invoices, accuracy on financial tables, accuracy on the mixed set, peak accuracy on its strongest document type, and consistency across all of them. Docling draws the widest shape on the structured axes and on peak accuracy, which reflects how well it reads invoices and tables, but it contracts on the mixed-document and consistency axes because its quality depends heavily on how predictable the layout is. PaddleOCR draws the most balanced shape of the four; it rarely reaches the outer edge on any single axis, yet it never collapses inward either, and that even profile is what makes it dependable when the input is unknown. LlamaParse reaches furthest on the mixed-document axis and stays strong on invoices, while giving back some ground on financial tables. Surya stays close to the centre on every axis, the visual signature of an engine that has not yet reached production quality on this workload.

Roughly seven in ten outputs pass, and close to three in ten need correction. Across all four June test sets and four engines, including inferred score-band verdicts for Invoice Run B, 72% of outputs landed at excellent or good. The remaining 28% carried a partial, poor or failed verdict, which is the review load any document pipeline has to plan around regardless of which engine leads on paper.

06 · BUSINESS IMPACT · JUNE BASELINES

On mixed documents, the wrong engine adds 300 human reviews per 1,000

Accuracy scores turn into a budget line the moment they meet real volume. Any document that does not come back clean goes to a person, and people are the expensive part of a document pipeline. That makes the pass rate easy to read as a review load, where every output below the pass bar is one more document that someone has to open, check, and correct by hand.

On a run of 10,000 mixed documents, the gap between the best and worst mainstream engine is roughly 3,000 extra documents in the review queue, produced by nothing more than picking the engine that tops the invoice leaderboard and assuming it travels. On financial tables the same logic favours PaddleOCR over LlamaParse by about 190 documents per 1,000. Surya falls below the quality bar on most outputs in these two samples.

June estimated review candidates per 1,000 outputs

Review load is the inverse of pass rate. Invoices are omitted because all mainstream engines clear them with almost no rework.

The saving comes from routing. Matching each document type to its strongest engine, rather than standardising on one, removes review load that a single fixed choice would quietly carry for the life of the pipeline.

07 · THE BIGGER PICTURE · JUNE BASELINES

OCR is the silent failure point in document AI

Most teams treat OCR as solved plumbing and spend their attention on the model at the end of the chain. The data argues the opposite. OCR sits at the front of every document-AI pipeline, and whatever it gets wrong is baked into everything downstream. A retrieval system cannot find a figure that was merged into the wrong cell, and a language model will answer confidently from text that was already garbled before it arrived.

The pipeline runs in one direction: Document → OCR (error introduced) → Chunk & index (error preserved) → Retrieve (wrong or missing context) → LLM answer (error amplified). An early mistake does not stay small; it travels.

The benchmark this report is built on, OHR-Bench, exists for exactly this reason: it measures how much OCR quality changes the answers a RAG system produces. A 0.69 OCR score does not translate into 69% usable answers, because the errors concentrate on the parts that carry meaning, the totals, dates, labels and table relationships. Those are the tokens a downstream model leans on most. This error amplification aligns with our earlier analysis of LLMs vs. cloud OCR.

This reframes what a team should be optimising for. The engine that matters is the one that preserves the structure and the numbers the rest of the stack depends on, which is a different target from raw character recognition. It also explains why the leader changed by document type in this study, and why a team building document AI is better off benchmarking OCR on its own documents than trusting a result built on someone else's.

08 · ENGINE-SELECTION GUIDE · JUNE BASELINES

Which engine to use for which documents

If GPU inference fits your workload, trial Unlimited-OCR first for the profiles tested here. The June primary for each document type remains a fallback candidate; validate accuracy, latency and review burden on your own intake.

The findings above translate into a short set of routing rules. Each document profile has a primary engine and a fallback, where the fallback takes over when the primary errors, times out, or returns output the pipeline flags as low confidence.

09 · ENGINE PROFILES · JUNE BASELINES

The five engines in detail

Unlimited-OCR · October accuracy leader

Profile. Highest mean judge score in all four evaluation sets, with the clearest gains on financial tables and mixed financial-report pages. Strong scores coexist with table-alignment issues and extra reference-mismatched page text.

Use it when. Extraction quality on the tested profiles matters, GPU inference is practical, and you can validate numbers and structure. Keep a fallback and measure end-to-end cost.

Docling structured-document specialist

Profile. It posts the highest peak accuracy of any engine on invoices and financial tables and produces the cleanest output on structured material. It is also the most fragile of the leaders, with a 0.30 drop from its easiest to its hardest set and a 38% pass rate on mixed documents.

Use it when. The layout is predictable and you control the document format.

PaddleOCR the all-rounder

Profile. It draws the flattest curve in the field, with the smallest fall across document types (0.16) and the best pass rate on both hard categories. It rarely tops a single category, and it never collapses in one.

Use it when. The document mix is unknown, or you want one dependable default across a varied pipeline.

LlamaParse the generalist

Profile. It records the highest average across all four test sets and is the only engine to win the hardest category. It is strong on invoices, weakest of the top three on financial tables, and prone to hallucinated fields that need a validation step.

Use it when. Documents are mixed or messy and a review step can catch invented or mis-mapped fields.

Surya OCR trailing

Profile. It places last in every test set, with the steepest fall (0.45) and pass rates of 7% and 2% on financial tables and mixed documents. It can read simple invoices but breaks down on structure.

Use it when. Lightweight or multilingual text, not dense tables.

10 · RECOMMENDATION · JUNE BASELINES

What to deploy across a document pipeline

• Trial Unlimited-OCR first when GPU inference is practical. It led the October judge scores and reduced the estimated mixed-page review queue from PaddleOCR's 320 per 1,000 to 60. These are verdict-based estimates, not measured labor savings.

• Route by document type rather than picking one engine. The averages across all four sets are nearly identical for the top three (0.864 to 0.868), which hides the real differences. Per-type routing captures 3 to 30 points of accuracy that a single choice would leave on the table.

• Docling for structured intake. Invoices, forms and financial tables go to Docling first, with LlamaParse as the fallback.

• LlamaParse or PaddleOCR for mixed intake. Free-form and unpredictable documents go to LlamaParse, with PaddleOCR as the reliable fallback. Docling should not run alone in this lane.

• PaddleOCR as the global default. When a pipeline cannot classify documents up front, PaddleOCR gives the most stable result across the full range.

• Validate the high-risk classes everywhere. Mixed documents raise the share of hallucination and numeric errors, so add automated checks on numeric fields, totals and table row and column counts regardless of engine.

• Surya OCR trailed on these tested tables and financial-report pages. Evaluate other versions and document types separately.

Methodology and caveats. 1,600 graded outputs across four test sets and four engines, each output scored 0–1 by an LLM-as-grader against fixed ground truth. Invoice Run B shipped without verdict or issue-type labels, so its verdicts were derived from score bands calibrated on the labelled runs (99.3% agreement with the grader), and it is excluded from issue-composition analysis. Absolute scores are most reliable within a test set; cross-set comparison uses relative ranking, because Invoice Run B was graded on a compressed scale. Dataset-to-industry mapping is indicative rather than a controlled industry sample. All figures were recomputed directly from the source files.

October methodology and limits. The complete study contains 1,999 completed scores from 300 source documents: 100 invoices, 100 FinTabNet tables and 100 mixed financial-report pages. Both invoice runs grade the same extraction outputs and reference answers. They are not independent samples. One October Invoice A row remained in platform scoring status and is excluded, despite its populated auxiliary score field.

The evaluation notes report the same GPT-5 judge and prompts, plus an explicit score passthrough rule required by the newer platform. A documented 20-row calibration check reports correlation 0.97 and October scores about 0.03 lower on average. That small check does not establish identical grading or justify adjusting the new scores upward.

The October mixed-page run normalizes curly quotes, long dashes and other special characters in output and reference to avoid encoding problems; the superseded garbled run is excluded. Product versions and all inference settings are not fully recorded. No confidence intervals, independent human audit or matched cost/latency comparison is included.

11 · HOW WE RAN IT · JUNE BASELINES

How we conducted the evaluation

Every score in this report was produced on Gaugetuple, our evaluation platform, rather than read off by a person. For each document, the grader received the engine's output and the fixed ground-truth answer, then returned a score from zero to one, a categorical verdict, and a tag for the kind of error when one was present. Grading this way keeps the judgement consistent across 1,600 outputs and makes every figure here traceable back to a specific graded row. The screens below are taken from the same runs the analysis is built on.

Result Explorer

Result Explorer, run #5eays (OHR-Bench): 400 graded outputs, 275 completed, 125 failed, with each row's input, golden answer and engine output side by side.

Evaluation Workspace

All three runs in one workspace: #5eays, #zs7hu and #40yz1, alongside the wider evaluation history across 14 linked datasets and 2,346 data rows.

Failure Analysis

One graded result in full: Docling on an OHR-Bench page scores 40% and fails, tagged poor with the grader's reasoning.

Evaluation Pipeline

The evaluation pipeline: golden datasets, evaluation datasets, scoring configurations and run history that produced every number in this report.

The October update used three new runs in the same workspace: #9ndgp, #0qcv6 and #944wj.

Gaugetuple October mixed-page run with 100 scored outputs and 98 threshold passes

October mixed-page run #944wj. The interface shows 98 completed and two failed threshold outcomes. All 100 rows have numeric scores; 94 have good or excellent verdicts. Threshold outcomes and quality verdicts describe different acceptance criteria.

Unlimited-OCR financial-table result in Gaugetuple scored 0.99 with an excellent verdict

One October financial-table result scored 0.99 and was rated excellent; its reasoning notes a minor spacing difference. This is an example, rather than a claim that every table is clean.

OUR POINT OF VIEW

Choose the OCR engine on evidence

This started as a comparison of four engines and turned into a reminder about a component most teams stop thinking about too early. The quality of OCR quietly sets the ceiling for everything built on top of it, and three principles guide how we treat that at Newtuple.

Evaluate on your own documents. Public leaderboards rank engines on documents that are not yours. The order changed across our own test sets, so the ranking worth acting on is the one measured on the documents your business actually processes.

Route by document type. A single engine for everything leaves both accuracy and review hours on the table. Matching the engine to the document type, with a reliable fallback behind it, is what separates a clean pipeline from a growing review queue.

Re-benchmark as a habit. Four months after our June comparison, a new engine led the mean judge scores in every evaluation set. Revisit the choice when models change, use consistent preprocessing, and compare the operating cost as well as extraction quality.

Newtuple builds and evaluates document-AI systems, and Gaugetuple is the platform we use to keep them honest. If you are choosing an OCR engine or grading a document pipeline, the same evaluation can run on your own documents. Start the conversation at newtuple.com.

Sources: Newtuple Gaugetuple exports from June 19-21 and October 14, 2026; Baidu's official Unlimited-OCR model card and repository; the supplied evaluation package's separate recall and timing summaries.

Download the aggregate benchmark data.