Planning Before Reasoning: A Vision-Language Dataset for Structured Stepwise Reasoning in Document Understanding

Shin, Dongjae; Yoo, Hangyeol; Nam, Yerin; Shin, Hoyeon; Song, SeungWoo; Song, Seohyun; Yang, Donghun; Lim, KyungTae

doi:10.18653/2026.findings-emnlp.996

DocPlan: 133K document VQA examples annotated with query analysis, planning, extraction-and-reasoning, and answer stages

Findings of EMNLP 2026

DongJae Shin^1,*, Hangyeol Yoo^2,*, Yerin Nam², Hoyeon Shin³, SeungWoo Song², Seohyun Song³, Donghun Yang⁴, KyungTae Lim²

¹TutorusLabs ²KAIST ³Seoultech ⁴KISTI

^*Equal contribution

Abstract

This paper introduces DocPlan, a vision-language training dataset designed to emulate the stepwise reasoning strategies employed by human during visual document understanding. DocPlan was constructed to enable vision-language models (VLMs) to effectively learn the structured reasoning process that humans typically follow when interpreting documents. We reorganized existing document understanding datasets into 133K examples annotated with the following four sequential stages: Recognition → Planning → Extraction and Reasoning → Conclusion. To evaluate the effectiveness of DocPlan, we fine-tuned the direct prediction-based VLMs to perform stepwise reasoning generation across these stages. The experimental results demonstrated that our approach achieved up to a 13.8%p improvement in accuracy on document-based benchmarks.

Keywords Visual document understanding Document VQA Chart question answering Vision-language models (VLMs) Multimodal reasoning Stepwise reasoning Chain-of-thought (CoT) Plan-and-reason Synthetic reasoning data Interpretability

Dataset Construction

Three-step DocPlan construction pipeline. Step 1: images from DocVQA, InfoVQA, and ChartQA are given to Qwen 2.5-VL, which generates queries of five types (CP, FP, LR, MA, IR) at Basic, Intermediate, and Advanced difficulty. Step 2: generated and original queries pass through Qwen2.5-VL 72B, which writes query analysis, definition of plan, execution of plan, and final answer, and the outputs are combined into the DocPlan dataset. Step 3: origin and generated samples go through query filtering, rule-based format filtering, and model agreement among GPT-4o, Qwen2.5-VL, and Gemini 2.0 to give the filtered DocPlan dataset.

Figure 3. Construction of the DocPlan dataset in three steps: (1) query generation, (2) plan and reasoning generation, and (3) data filtering. The pipeline starts from about 28K training images of DocVQA, InfoVQA, and ChartQA and 162K queries (82K original, 80K generated), and keeps 133,623 samples after filtering.

Existing document VQA datasets mostly pair an image with a short answer, so a model trained on them learns direct prediction and can reach the right answer through wrong intermediate reasoning (Figure 1 of the paper shows such a case on ChartQA). DocPlan follows the order in which people read charts and documents (recognize the question, plan, extract values, reason, conclude) and writes it out as explicit tagged stages. It is built as follows:

Image collection. All training images of DocVQA, InfoVQA, and ChartQA are reused unchanged (about 28K images; 28,954 in the final dataset).
Query generation. Because original queries are mostly simple extraction questions, Qwen2.5-VL generates 80K new queries of five document-specific types adapted from MMStar (Fine-grained Perception, Coarse Perception, Instance Reasoning, Logical Reasoning, Mathematics) at three difficulty levels (Basic, Intermediate, Advanced). With the 82K original queries this gives 162K queries.
Plan-and-reason generation. For every query, Qwen2.5-VL writes a query analysis, then a plan with N sub-plans (N chosen by query type and difficulty), then executes each sub-plan in turn by extracting the needed information and reasoning over it, reusing earlier intermediate answers, and finally writes the final answer.
Filtering. Original queries (with ground truth) pass a rule-based tag check, then GPT-4o, Qwen2.5-VL, and Gemini 2.0 rate each sample and only samples on which the models agree are kept. Generated queries (without ground truth) are first checked by GPT-4o for validity against the image, then pass the same tag check and model-agreement filtering.

The result is DocPlan with 133,623 samples (68,513 from original queries and 65,110 from generated queries). An extra trace-based “hard filtering” step with human review of 10,207 likely errors produces DocPlan-HF with 124,053 samples. Both are released on Hugging Face (DocPlan-133K, DocPlan-HF-124K; access is granted on request). VLMs are fine-tuned on the data with visual instruction tuning for three epochs and evaluated with VLMEvalKit on DocVQA and InfoVQA (ANLS) and ChartQA and ChartQAPro (GPT-4o as judge).

Main Results

Up to 13.8%p higher average accuracy. Training paligemma2-10B on DocPlan raises its average over DocVQA, InfoVQA, ChartQA, and ChartQAPro from 40.1 to 53.9; Llama3.2-11B-Vision goes from 52.3 to 69.2 (59.3 when trained on the original QA data without reasoning).
Planning beats direct prediction on the same data. Rewriting the original 82K QA pairs into the DocPlan format (+DocPlan(origin)) improves Llama3.2 by 7.1%p (59.3 → 66.4) and gemma3 by 2.4%p (58.6 → 61.0) over direct training; adding the generated queries gives Llama3.2 another 2.8%p (69.2).
Structured stages beat free-form chain-of-thought. With the same base models, DocPlan outperforms free-form CoT, e.g. Gemma3-4b DocVQA 82.8 vs. 53.46 and ChartQA 81.3 vs. 54.0, and Llama-Vision InfoVQA 71.2 vs. 54.83. In the stage ablation, adding the execution stage gives the largest gain (5.3%p).
Competitive with larger systems and transfers to held-out data. Llama-3.2-11B+DocPlan reaches 91.5 on DocVQA and 86.6 on ChartQA, above Llava-CoT (89.8 / 82.6) and zero-shot GPT-4o (85.6 / 85.7). On held-out TextVQA, Gemma3-4B-it improves from 58.14 to 62.91 (+4.77%p).
More interpretable reasoning traces. In a simulatability test (answering from the reasoning trace alone, without the image), Llama3.2+DocPlan scores 62.0 on average vs. 51.5 for Llava-CoT (+10.5%p); in GPT pairwise preference it wins 71.3% of comparisons on average vs. 26.1% for Llava-CoT.

Trade-offs reported in the paper: responses become much longer (e.g. Llama3.2-11B-V generates 177.5–285.3 tokens on average instead of 5.7–26.1), and accuracy on the general-domain MMMU benchmark drops after document-only fine-tuning, though less than with direct-prediction training (Llama3.2-11B-Vision: −15.0%p vs. −31.2%p).

**Table 1.** Accuracy of five open VLMs under different training data. Base: no fine-tuning; +origin dataset: 82K original QA pairs without reasoning; +DocPlan(origin): the 68K original queries converted to DocPlan format; +DocPlan: those plus 65K generated queries. DocVQA and InfoVQA use ANLS, ChartQA and ChartQAPro use GPT-4o-judged accuracy. Bold marks the best value per model, as in the paper. (Table 2 in the paper.)
Model	Training data	DocVQA	InfoVQA	ChartQA	ChartQAPro	Avg
SmolVLM2-2.2B	Base	80.9	47.6	66.1	17.8	53.1
	+origin dataset	77.0	45.1	63.3	18.3	50.9
	+DocPlan(origin)	75.8	43.8	70.8	17.1	51.9
	+DocPlan	80.1	44.4	72.7	18.9	54.0
gemma-3-4b-it	Base	62.0	30.1	57.4	21.6	42.8
	+origin dataset	83.7	50.3	76.1	24.1	58.6
	+DocPlan(origin)	82.7	52.9	81.0	27.3	61.0
	+DocPlan	82.8	50.4	81.3	28.9	60.9
Qwen2-VL-7B	Base	93.1	72.7	83.1	26.6	68.9
	+origin dataset	89.2	66.0	78.6	23.1	64.2
	+DocPlan(origin)	91.8	73.4	79.9	29.9	68.8
	+DocPlan	91.8	71.2	83.2	30.0	69.0
paligemma2-10B	Base	69.0	34.9	34.2	22.4	40.1
	+origin dataset	71.9	39.5	67.6	16.6	48.9
	+DocPlan(origin)	74.9	42.3	71.5	20.7	52.3
	+DocPlan	76.4	42.3	74.5	22.4	53.9
Llama3.2-11B-V	Base	83.0	41.0	74.2	11.0	52.3
	+origin dataset	87.8	59.5	73.4	16.5	59.3
	+DocPlan(origin)	90.4	68.3	79.8	27.1	66.4
	+DocPlan	91.5	68.2	86.6	30.6	69.2

@inproceedings{shin-etal-2026-planning, title = "Planning Before Reasoning: A Vision-Language Dataset for Structured Stepwise Reasoning in Document Understanding", author = "Shin, Dongjae and Yoo, Hangyeol and Nam, Yerin and Shin, Hoyeon and Song, SeungWoo and Song, Seohyun and Yang, Donghun and Lim, KyungTae", editor = "Lee, Hung-Yi and Dev, Sunipa and Li, Jessy and Elliott, Desmond", booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {EMNLP} 2026", month = oct, year = "2026", address = "Budapest, Hungary", publisher = "Association for Computational Linguistics", url = "https://aclanthology.org/2026.findings-emnlp.996/", doi = "10.18653/2026.findings-emnlp.996", pages = "21022--21061", ISBN = "979-8-89176-462-0" }

Planning Before Reasoning: A Vision-Language Dataset for Structured Stepwise Reasoning in Document Understanding

Abstract

Dataset Construction

Main Results

BibTeX