FinSpread-Bench

We found that the major frontier models exceed human performance in financial spreading: GPT-5.2, Gemini 3.1 Pro, and Opus 4.6 all comfortably land above 90% match rate for this task. However, the latest model generation is necessary to achieve this. Smaller and older models such as Haiku are not sufficient for this complex task. This finding supports the conclusion that AI has crossed a meaningful performance boundary in late 2025, which is now allowing to automate harder, more ambiguous tasks than before.

Across the test matrix, we see meaningful variance in accuracy across configurations:

Accuracy is strongly dependent on the reasoning model. Moving away from the latest frontier models leads to rapid performance degradation (e.g., comparing the results of Gemini 3.1 Pro vs. Gemini 2.5 Pro as the reasoning model). This shows that failures mostly stem from interpretation and judgment rather than incorrect extractions from the source documents. In practice, this means frontier reasoning models are currently required to reach the match rate levels reported here, and this level of performance has only become feasible very recently as these models have improved.

We also see a clear model size effect within reasoning models: smaller models (e.g., Claude Haiku vs. Sonnet) underperform in large part due to schema reliability rather than incorrect values. In our runs, 59.5% of Haiku outputs and 38.1% of Sonnet outputs failed to produce a complete schema. Conditional on producing a complete schema, match rates were 84.3% (Haiku) and 88.2% (Sonnet). These conditional figures should be treated cautiously because schema failures concentrate in the most complex cases; if those cases succeeded, average match rate would likely go down as well.

On the document extraction side, Gemini 2.5 Pro already seems to deliver comparable results to Gemini 3.1 Pro. Both Flash models appear more inconsistent in extraction accuracy today, implying a tradeoff between cost/latency and accuracy. Notably, the remaining extraction inaccuracies are usually not simple “number typos”, but failures to correctly interpret which period a table view refers to, leading to values being allocated to the wrong period. We expect the faster models to keep improving, and we hope they eventually become strong enough to reduce this tradeoff so we can default to the faster and cheaper variants more often.

To better understand performance drivers, we separate the evaluation into two sub-tasks: multi-period spreading, which extracts and harmonizes financial data across multiple periods (e.g., fiscal year 2023 and fiscal year 2024) to enable cross-period comparison, and year-to-date (YTD) spreading, which focuses on spreading and harmonizing year-to-date financials for in-period monitoring and decision-making.

YTD spreading is the harder task. Unlike annual financial statements that typically report complete periods, YTD packages often contain partial months or a mix of monthly and cumulative views, and the “correct” YTD total may need to be inferred rather than read directly: the agent must (i) correctly interpret which time window each table represents, (ii) reconcile inconsistent period cutoffs across statements, and (iii) compute missing totals by summing the right months while avoiding double counting when both monthly and YTD figures are present (see complex example in the introduction). These steps introduce more implicit logic and more opportunities for period misalignment, which is why YTD accuracy is more sensitive to reasoning and tooling support.

With respect to tooling, the largest remaining failure mode is numerical aggregation, especially when values must be combined across periods. Without reliable math support, accuracy degrades substantially for complex cases. This effect is much stronger in the YTD documents than in the multi-period documents, since YTD spreading is the most computation-heavy, whereas in AFS spreading many fields can be extracted as-is or only require simpler additions and subtractions.

Observed accuracy differences typically decompose into three types of error: (i) extraction errors where the source numbers are captured incorrectly, (ii) reasoning or mapping errors where the correct number is present but assigned to the wrong field or treated with the wrong convention, and (iii) computation errors where values must be calculated or aggregated across periods. When using the most powerful configuration in our test matrix, the remaining mistakes are dominated by (ii), mapping and convention errors. In particular, debt classification (e.g., short-term vs. long-term debt and current vs. non-current portions) requires nuanced judgement, often relying on institution-specific rules about how to map specific debt instruments and disclosure patterns into standardized fields. As a result, achieving near-perfect adherence typically requires iteration on these mapping nuances as edge cases are discovered.

We do not discuss cost and latency in detail here since both metrics are orders of magnitude below the human baseline. Completing the task currently takes 5–15 minutes and we expect this to drop significantly over the next six months as new models become available.