Instructions to use zenosai/MonkeyOCRv2-B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use zenosai/MonkeyOCRv2-B with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-feature-extraction", model="zenosai/MonkeyOCRv2-B", trust_remote_code=True)# Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("zenosai/MonkeyOCRv2-B", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- News
- Use MonkeyOCRv2 as a Vision Backbone
- MonkeyDoc v2
- Model Zoo
- Quick Start
- Vision Encoder
- Document Parsing
- Document Understanding
- 1. Install
- 2. Download Model Weights
- 3. Inference
- 1. Text recognition results on Common Benchmarks, Union14M-Benchmark, OST, and Chinese Benchmarks. We follow the training and evaluation protocols of OpenOCR.
- 2. Formula recognition results on OmniDocBench 1.6, MathWriting, and UniMER-Test.
- 3. Text detection results on Total-Text, CTW1500, ICDAR2015 and ArT. We follow the training and evaluation protocols of MMOCR and DPText-DETR.
- 4. Document tampering detection results on DocTamper benchmark.
- 5. Overlapping text segmentation results on MOT dataset.
- 6. Document parsing results on MDPBench, a comprehensive multilingual benchmark for real-world document parsing.
- 7. Document understanding performance comparison across different vision foundation models. The evaluation benchmarks are selected following TextMonkey and DT-VQA.
- Visualization
- 1. Text recognition results on Common Benchmarks, Union14M-Benchmark, OST, and Chinese Benchmarks. We follow the training and evaluation protocols of OpenOCR.
- 2. Formula recognition results on OmniDocBench 1.6, MathWriting, and UniMER-Test.
- 3. Text detection results on Total-Text, CTW1500, ICDAR2015 and ArT. We follow the training and evaluation protocols of MMOCR and DPText-DETR.
- 4. Document tampering detection results on DocTamper benchmark.
- 5. Overlapping text segmentation results on MOT dataset.
- 6. Document parsing results on MDPBench, a comprehensive multilingual benchmark for real-world document parsing.
- 7. Document understanding performance comparison across different vision foundation models. The evaluation benchmarks are selected following TextMonkey and DT-VQA.
- 1. Text recognition results on Common Benchmarks, Union14M-Benchmark, OST, and Chinese Benchmarks. We follow the training and evaluation protocols of OpenOCR.
- Evaluation Results
- 1. Text recognition results on Common Benchmarks, Union14M-Benchmark, OST, and Chinese Benchmarks. We follow the training and evaluation protocols of OpenOCR.
- 2. Formula recognition results on OmniDocBench 1.6, MathWriting, and UniMER-Test.
- 3. Text detection results on Total-Text, CTW1500, ICDAR2015 and ArT. We follow the training and evaluation protocols of MMOCR and DPText-DETR.
- 4. Document tampering detection results on DocTamper benchmark.
- 5. Overlapping text segmentation results on MOT dataset.
- 6. Document parsing results on MDPBench, a comprehensive multilingual benchmark for real-world document parsing.
- 7. Document understanding performance comparison across different vision foundation models. The evaluation benchmarks are selected following TextMonkey and DT-VQA.
- 1. Text recognition results on Common Benchmarks, Union14M-Benchmark, OST, and Chinese Benchmarks. We follow the training and evaluation protocols of OpenOCR.
- MonkeyDoc v2
- Citation
- Copyright
News
2026.07.21📦 We release MonkeyDoc v2, an open multilingual corpus for document-oriented pretraining. We hope it can serve as a shared data foundation for more transparent, reproducible, and fair comparisons in Document AI.2026.07.11🚀 We release MonkeyOCRv2, including MonkeyOCRv2 vision encoder, MonkeyOCRv2-Parsing for multilingual document parsing, MonkeyOCRv2-Und for efficient document understanding.
Use MonkeyOCRv2 as a Vision Backbone
MonkeyOCRv2 is released as a standalone, document-native vision encoder. It can be integrated into different OCR and document AI systems as a visual backbone.
The current release has been evaluated on document parsing, document understanding, text recognition, formula recognition, text detection, document tampering detection, and overlapping-text segmentation.
Beyond these evaluated tasks, the encoder may also be useful for text-rich scenarios such as scientific papers, historical documents, medical reports, charts and tables, and remote-sensing maps or reports. We welcome community exploration of these directions.
from transformers import AutoModel
encoder = AutoModel.from_pretrained(
"zenosai/MonkeyOCRv2-B",
trust_remote_code=True,
dtype="auto",
device_map="auto",
)
See the Vision Encoder Quick Start for installation and feature-extraction examples. If you adapt MonkeyOCRv2 to a new task or domain, feel free to open an issue or pull request and share the results.
MonkeyDoc v2
MonkeyDoc v2 is currently the largest document image pre-training image-text pair dataset, comprising 113 million document images across 17 languages. The open-sourcing of MonkeyDoc v2 is still underway. So far, we have released all 52 million synthetic samples and 35 million real-world samples derived from FinePDF. Additional real-world data from other sources is currently being organized and will be released progressively. You can download the full datset as follows:
pip install modelscope
modelscope download --dataset zenosai/MonkeyDocv2 --local_dir ./MonkeyDocv2
Model Zoo
1. Vision Encoder
| Model | Backbone | Params | Pretraining Resolution |
Applicable Tasks | Checkpoint Link |
|---|---|---|---|---|---|
| Monkey OCRv2-S | ViT-S | 28M | 1280*28*28 | Recognition / Parsing / Understanding | 🤗HuggingFace 🤖ModelScope |
| Monkey OCRv2-B | ViT-B | 113M | 1280*28*28 | Recognition / Parsing / Understanding | 🤗HuggingFace 🤖ModelScope |
| Monkey OCRv2-AS | ViTAEv2-S | 21M | 1760*32*32 | Detection / Segmentation | 🤗HuggingFace 🤖ModelScope |
2. Document Parsing Model
| Model | Link | Total Params | ViT | LLM | All | Digit. | Photo. | Latin Avg. | DE | EN | ES | FR | ID | IT | NL | PT | VI | Non-Latin Avg. | AR | HI | JP | KO | RU | TH | ZH | ZH-T |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| MonkeyOCRv2-S-Parsing | HuggingFace ModelScope | 0.6B | 0.03B | 0.6B | 82.5 | 87.9 | 80.7 | 83.2 | 87.3 | 83.6 | 76.8 | 73.6 | 85.4 | 87.2 | 85.5 | 87.4 | 81.9 | 81.7 | 91.2 | 87.1 | 69.9 | 88.7 | 78.0 | 79.8 | 84.4 | 74.7 |
| MonkeyOCRv2-B-Parsing | HuggingFace ModelScope | 0.7B | 0.1B | 0.6B | 83.3 | 88.1 | 81.7 | 84.2 | 87.7 | 84.5 | 75.2 | 78.4 | 86.5 | 88.6 | 86.1 | 87.9 | 83.2 | 82.1 | 90.7 | 87.2 | 71.9 | 87.6 | 80.1 | 80.8 | 83.6 | 75.3 |
3. Document Understanding Model
| Model | Link | Total Params | Overall | DocVQA | InfoVQA | DF | KLC | WTQ | ChartQA | DT-VQA | OCRBench |
|---|---|---|---|---|---|---|---|---|---|---|---|
| MonkeyOCRv2-S-Und | HuggingFace ModelScope | 1.7B | 55.9 | 79.3 | 44.5 | 65.1 | 37.6 | 43.0 | 62.0 | 63.1 | 52.2 |
| MonkeyOCRv2-B-Und | HuggingFace ModelScope | 1.8B | 57.2 | 79.3 | 46.3 | 65.8 | 38.2 | 43.2 | 62.0 | 64.3 | 58.1 |
Quick Start
Vision Encoder
1. Install
Install transformers and flash attention:
conda create -n MonkeyOCRv2 python=3.10
conda activate MonkeyOCRv2
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu126
pip install transformers==4.57.6
pip install flash-attn==2.7.4.post1 --no-build-isolation
pip install accelerate
pip install qwen_vl_utils
2. Download Model Weights
Download our model from Huggingface.
python download_model.py -n MonkeyOCRv2-B # or MonkeyOCRv2-S / MonkeyOCRv2-AS
You can also download our model from ModelScope.
pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-B # or MonkeyOCRv2-S / MonkeyOCRv2-AS
3. Extract Image Feature
cd vision
# For MonkeyOCRv2-B and MonkeyOCRv2-S
python extract_feature.py
# For MonkeyOCRv2-AS
python extract_feature_vitae.py
Document Parsing
1. Install
Install vLLM following its official guide:
conda create -n MonkeyOCRv2Parsing python=3.10
conda activate MonkeyOCRv2Parsing
pip install uv
uv pip install vllm==0.11.2 --torch-backend=auto -i https://pypi.tuna.tsinghua.edu.cn/simple requests
pip install -r parsing/requirements.txt
2. Download Model Weights
Download our model from Huggingface.
python download_model.py -n MonkeyOCRv2-B-Parsing # or MonkeyOCRv2-S-Parsing
You can also download our model from ModelScope.
pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-B-Parsing # or MonkeyOCRv2-S-Parsing
3. vLLM Serving
You should start a vLLM service before parsing documents:
cd parsing
python serve.py -m ../model_weight/MonkeyOCRv2-B-Parsing -p 8888
# Show help messages
python serve.py -h
4. Inference
You can parse documents using CLI or serve with demo and FastAPI.
4.1 Parse using CLI
Parse a single document or a directory containing PDFs or images:
cd parsing
python parse.py \
-i ../images_test \
-o output/test \
-s http://127.0.0.1:8888 \
--draw-layout \
--skip-processed
# Show help messages
python parse.py -h
4.2 Serve with Web Demo
cd parsing
python demo/gradio_demo.py -s http://127.0.0.1:8888 -p 8891
# Show help messages
python demo/gradio_demo.py -h
You can access the web demo at http://localhost:8891.
4.3 Serve with FastAPI
cd parsing
python fastapi/main.py -s http://127.0.0.1:8888 -p 8000
# Show help messages
python fastapi/main.py -h
You can access the API documentation at http://localhost:8000/docs to explore available endpoints.
Document Understanding
1. Install
See install part of MonkeyOCRv2.
2. Download Model Weights
Download our model from Huggingface.
python download_model.py -n MonkeyOCRv2-B-Und # or MonkeyOCRv2-S-Und
You can also download our model from ModelScope.
pip install modelscope
python download_model.py -t modelscope -n MonkeyOCRv2-B-Und # or MonkeyOCRv2-S-Und
3. Inference
cd understanding
python infer.py \
-m ../model_weight/MonkeyOCRv2-B-Und \
-i ../images_test/vqa.png \
-q 'What is the serving size?'
# Show help messages
python infer.py -h
Visualization
Our model supports robust document parsing in real-world scenarios across 17 languages, including Simplified Chinese (ZH), Traditional Chinese (ZH-T), English (EN), Arabic (AR), German (DE), Spanish (ES), French (FR), Hindi (HI), Indonesian (ID), Italian (IT), Japanese (JP), Korean (KO), Dutch (NL), Portuguese (PT), Russian (RU), Thai (TH), and Vietnamese (VI).
Evaluation Results
1. Text recognition results on Common Benchmarks, Union14M-Benchmark, OST, and Chinese Benchmarks. We follow the training and evaluation protocols of OpenOCR.
| Model | Overall | Union14M-Benchmark | Chinese Benchmarks | Occlusion SceneText | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Avg | Artistic | Context less | Curve | General | Multi Oriented | Multi Words | Saliency | Avg | Scene | Web | Document | Hand writing | |||
| ABINet | 73.7 | 75.7 | 71.7 | 74.7 | 80.4 | 79.8 | 69.0 | 76.8 | 77.6 | 70.3 | 66.6 | 63.2 | 98.2 | 53.1 | 75.0 |
| MAERec | 81.6 | 85.2 | 79.0 | 84.2 | 89.1 | 84.6 | 87.1 | 85.9 | 86.3 | 83.1 | 84.4 | 83.0 | 99.5 | 65.6 | 76.4 |
| CPPD | 80.4 | 81.9 | 76.5 | 82.9 | 86.2 | 83.5 | 78.7 | 81.9 | 83.5 | 81.7 | 82.7 | 82.4 | 99.4 | 62.3 | 79.6 |
| IGTR-AR | 81.0 | 84.9 | 77.0 | 82.4 | 90.4 | 84.4 | 91.2 | 84.0 | 84.7 | 81.7 | 82.0 | 81.7 | 99.5 | 63.8 | 76.3 |
| SMTR | 80.4 | 85.0 | 76.8 | 83.9 | 89.1 | 83.7 | 87.7 | 89.3 | 84.6 | 82.7 | 83.4 | 83.0 | 99.3 | 65.1 | 73.5 |
| SVTRv2 | 83.1 | 86.1 | 79.3 | 86.1 | 90.6 | 85.1 | 89.0 | 86.7 | 86.2 | 83.3 | 83.5 | 83.3 | 99.5 | 67.0 | 80.0 |
| CRNN (ResNet) | 58.7 | 49.2 | 51.2 | 62.3 | 48.1 | 68.2 | 13.0 | 60.4 | 41.4 | 68.8 | 63.8 | 68.2 | 97.0 | 46.1 | 58.0 |
| CRNN (MonkeyOCRv2-S) | 67.3 | 65.2 | 63.7 | 73.0 | 71.1 | 74.5 | 28.6 | 72.1 | 73.4 | 74.2 | 73.0 | 74.9 | 96.9 | 51.8 | 62.4 |
| PARSeq (ViT) | 82.2 | 84.3 | 76.5 | 83.4 | 87.6 | 84.9 | 88.8 | 84.3 | 84.4 | 82.4 | 84.2 | 82.8 | 99.5 | 63.0 | 79.9 |
| PARSeq (MonkeyOCRv2-S) | 84.3 | 87.6 | 78.6 | 86.4 | 92.1 | 85.4 | 93.9 | 88.7 | 87.7 | 83.7 | 84.6 | 83.2 | 99.5 | 67.3 | 81.5 |
2. Formula recognition results on OmniDocBench 1.6, MathWriting, and UniMER-Test.
| Model | Params | Overall | OmniDocBench 1.6 | MathWriting | SPE | CPE | HWE | SCE | |||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| CDM | ExpRate | CDM | ExpRate | CDM | ExpRate | CDM | ExpRate | CDM | ExpRate | CDM | ExpRate | CDM | ExpRate | ||
| Pix2tex | 25.5M | 53.8 | 23.3 | 69.4 | 27.0 | 0.4 | 0.0 | 96.2 | 72.4 | 64.9 | 7.1 | 24.5 | 0.6 | 67.6 | 32.8 |
| Texify | 312M | 67.3 | 40.4 | 76.5 | 46.4 | 26.6 | 2.0 | 98.5 | 91.0 | 70.4 | 28.2 | 52.7 | 23.6 | 79.3 | 51.3 |
| UniMERNet-B | 325M | 89.5 | 64.5 | 90.4 | 59.5 | 63.8 | 12.3 | 99.1 | 93.3 | 96.0 | 80.5 | 94.0 | 64.3 | 93.7 | 77.0 |
| UniMERNet-S | 202M | 89.8 | 64.0 | 90.1 | 59.1 | 65.9 | 12.7 | 99.1 | 93.4 | 95.9 | 77.7 | 93.7 | 63.9 | 94.1 | 76.9 |
| UniMERNet-T (Swin) | 107M | 89.4 | 61.8 | 89.9 | 57.2 | 65.6 | 12.9 | 99.1 | 92.3 | 94.9 | 69.9 | 93.3 | 61.9 | 93.8 | 76.6 |
| UniMERNet-T (MonkeyOCRv2-S) | 110M | 90.9 | 66.4 | 90.8 | 61.1 | 70.8 | 16.2 | 99.2 | 93.8 | 96.1 | 79.2 | 94.3 | 69.5 | 94.0 | 78.6 |
3. Text detection results on Total-Text, CTW1500, ICDAR2015 and ArT. We follow the training and evaluation protocols of MMOCR and DPText-DETR.
4. Document tampering detection results on DocTamper benchmark.
| Method | Params | Overall | DocTamper-Test | DocTamper-FCD | DocTamper-SCD | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| IoU | F | IoU | P | R | F | IoU | P | R | F | IoU | P | R | F | ||
| PSCC-Net | 5M | 13.7 | 31.3 | 17.0 | 25.0 | 83.0 | 39.0 | 13.0 | 19.0 | 82.0 | 30.0 | 11.0 | 15.0 | 83.0 | 25.0 |
| UperNet | 67M | 49.3 | 54.0 | 70.0 | 66.0 | 60.0 | 62.0 | 30.0 | 57.0 | 35.0 | 43.0 | 48.0 | 57.0 | 58.0 | 57.0 |
| CAT-Net | 114M | 67.3 | 71.0 | 78.0 | 75.0 | 69.0 | 72.0 | 66.0 | 85.0 | 70.0 | 76.0 | 58.0 | 65.0 | 65.0 | 65.0 |
| Swin-UPer | 81M | 66.7 | 71.7 | 79.0 | 75.0 | 72.0 | 73.0 | 64.0 | 80.0 | 70.0 | 75.0 | 57.0 | 66.0 | 68.0 | 67.0 |
| SegFormer | 85M | 70.3 | 74.0 | 81.0 | 77.0 | 74.0 | 75.0 | 69.0 | 82.0 | 74.0 | 78.0 | 61.0 | 68.0 | 70.0 | 69.0 |
| Mask2Former | 69M | 69.7 | 78.0 | 84.0 | 82.0 | 83.0 | 82.0 | 66.0 | 81.0 | 75.0 | 78.0 | 59.0 | 70.0 | 79.0 | 74.0 |
| ConvNext | 122M | 69.7 | 75.3 | 84.0 | 81.0 | 78.0 | 79.0 | 62.0 | 76.0 | 71.0 | 74.0 | 63.0 | 71.0 | 74.0 | 73.0 |
| ConvNextV2 | 121M | 72.7 | 77.7 | 86.0 | 82.0 | 79.0 | 81.0 | 65.0 | 79.0 | 75.0 | 77.0 | 67.0 | 74.0 | 76.0 | 75.0 |
| InternImage | 128M | 73.3 | 77.7 | 84.0 | 81.0 | 77.0 | 79.0 | 72.0 | 83.0 | 79.0 | 81.0 | 64.0 | 73.0 | 74.0 | 73.0 |
| ASC-Former | 80M | 68.2 | 80.8 | 81.5 | 91.8 | 87.8 | 89.8 | 61.3 | 74.9 | 77.1 | 76.0 | 61.9 | 78.0 | 75.0 | 76.5 |
| DTD | 66M | 77.0 | 79.7 | 84.0 | 81.0 | 77.0 | 79.0 | 79.0 | 88.0 | 82.0 | 85.0 | 68.0 | 75.0 | 76.0 | 75.0 |
| FFDN* (ViTAEv2) | 69M | 70.7 | 82.7 | 69.4 | 76.2 | 88.7 | 82.0 | 79.0 | 92.5 | 84.4 | 88.3 | 63.6 | 79.1 | 76.5 | 77.8 |
| FFDN (MonkeyOCRv2-AS) | 71M | 78.2 | 87.5 | 87.4 | 94.8 | 91.8 | 93.3 | 79.9 | 90.4 | 87.4 | 88.9 | 67.2 | 81.0 | 79.8 | 80.4 |
* denotes models trained with the ViTAEv2 pretrained by DeepSolo
5. Overlapping text segmentation results on MOT dataset.
| Model | mIoUText | IoUOcc | IoUOccd | IoUOv |
|---|---|---|---|---|
| Unet | 62.2 | 80.2 | 65.7 | 40.7 |
| Deeplab v3 | 67.9 | 83.2 | 71.2 | 49.3 |
| OCRNet | 65.8 | 81.0 | 68.5 | 47.8 |
| Segformer | 69.0 | 83.6 | 74.1 | 49.3 |
| MaskFormer | 68.4 | 83.5 | 70.3 | 51.4 |
| TexRNet | 68.9 | 84.2 | 73.2 | 49.3 |
| EAFormer | 69.1 | 83.8 | 74.2 | 50.5 |
| WASNet | 70.8 | 84.8 | 74.4 | 53.1 |
| Mask2Former (ResNet) | 70.3 | 84.7 | 73.3 | 52.8 |
| Mask2Former (MonkeyOCRv2-AS) | 76.6 | 88.6 | 83.4 | 57.7 |
| MOTS (ResNet) | 72.6 | 85.2 | 77.5 | 54.9 |
| MOTS (MonkeyOCRv2-AS) | 76.9 | 88.6 | 82.6 | 59.4 |
6. Document parsing results on MDPBench, a comprehensive multilingual benchmark for real-world document parsing.
| Model | Total Params | ViT | LLM | All | Digit. | Photo. | Latin Avg. | DE | EN | ES | FR | ID | IT | NL | PT | VI | Non-Latin Avg. | AR | HI | JP | KO | RU | TH | ZH | ZH-T |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Closed-source VLMs | |||||||||||||||||||||||||
| ChatGPT-5.2-2025-12-11 | - | - | - | 68.6 | 85.6 | 63.0 | 75.2 | 70.8 | 79.4 | 71.4 | 60.0 | 77.7 | 78.5 | 71.6 | 85.0 | 82.1 | 61.1 | 64.9 | 63.4 | 55.8 | 65.4 | 60.7 | 63.8 | 56.3 | 58.7 |
| Claude-Sonnet-4.6 | - | - | - | 73.1 | 85.0 | 69.3 | 79.2 | 79.8 | 80.6 | 72.8 | 66.5 | 82.3 | 83.3 | 76.7 | 88.0 | 83.1 | 66.2 | 67.8 | 71.7 | 63.4 | 64.3 | 70.8 | 65.2 | 61.3 | 65.1 |
| Doubao-2.0-pro | - | - | - | 74.2 | 78.9 | 72.8 | 75.7 | 82.8 | 74.4 | 69.0 | 70.0 | 73.3 | 82.0 | 69.9 | 83.4 | 76.5 | 72.5 | 81.3 | 75.7 | 65.8 | 74.7 | 63.3 | 71.9 | 71.9 | 75.2 |
| Gemini-3-pro | - | - | - | 86.4 | 90.4 | 85.1 | 88.4 | 91.2 | 90.6 | 83.4 | 82.7 | 91.5 | 91.6 | 87.7 | 91.4 | 85.9 | 84.1 | 89.4 | 90.4 | 74.8 | 85.5 | 84.9 | 80.6 | 85.1 | 82.1 |
| Open-source VLMs | |||||||||||||||||||||||||
| InternVL-3.5-8B | 8.3B | 0.3B | 8B | 42.7 | 59.7 | 37.0 | 53.4 | 39.8 | 64.2 | 47.5 | 42.7 | 53.8 | 60.6 | 52.2 | 63.2 | 57.0 | 30.6 | 8.2 | 9.0 | 45.6 | 30.3 | 26.1 | 10.8 | 55.3 | 59.3 |
| MinerU-2.5 | 1.2B | 0.7B | 0.5B | 46.3 | 61.9 | 40.8 | 63.0 | 68.8 | 78.4 | 54.7 | 57.3 | 67.5 | 75.2 | 60.4 | 58.8 | 46.0 | 27.4 | 1.3 | 9.0 | 39.1 | 14.7 | 8.6 | 11.3 | 72.9 | 62.2 |
| DeepSeek-OCR | 3.4B | 0.4B | 3B | 51.8 | 80.7 | 42.2 | 54.5 | 55.0 | 58.3 | 44.1 | 43.2 | 60.9 | 69.3 | 52.4 | 53.0 | 54.1 | 48.9 | 56.9 | 52.2 | 49.1 | 28.2 | 36.2 | 49.4 | 59.7 | 59.2 |
| MonkeyOCR-pro-3B | 3.7B | 0.7B | 3B | 52.2 | 68.0 | 47.0 | 65.1 | 71.7 | 77.9 | 55.9 | 62.1 | 66.2 | 74.5 | 66.3 | 71.1 | 40.2 | 37.6 | 4.6 | 4.2 | 55.2 | 60.5 | 42.6 | 9.1 | 72.2 | 52.4 |
| Nanonets-OCR-s | 4.7B | 0.7B | 4B | 63.7 | 78.8 | 58.7 | 71.3 | 75.1 | 78.5 | 61.2 | 62.5 | 70.3 | 81.0 | 69.6 | 75.9 | 67.5 | 55.0 | 59.5 | 61.8 | 55.9 | 51.2 | 43.5 | 39.5 | 67.4 | 61.5 |
| Nanonets-OCR2-3B | 3.7B | 0.7B | 3B | 64.2 | 79.2 | 59.3 | 71.4 | 76.7 | 76.4 | 61.8 | 66.1 | 68.4 | 78.5 | 74.1 | 74.2 | 66.0 | 56.2 | 60.2 | 59.2 | 52.1 | 54.7 | 45.5 | 44.6 | 68.3 | 65.1 |
| Qwen3.5-Instruct-9B | 9.7B | 0.7B | 9B | 65.7 | 74.8 | 62.7 | 72.5 | 72.8 | 72.0 | 72.0 | 64.4 | 66.2 | 77.6 | 74.5 | 79.1 | 74.0 | 58.2 | 53.4 | 56.2 | 55.7 | 60.3 | 54.7 | 56.7 | 60.8 | 67.5 |
| GLM-OCR | 0.9B | 0.4B | 0.5B | 67.3 | 77.9 | 63.7 | 78.7 | 82.7 | 84.5 | 75.8 | 76.2 | 79.7 | 82.8 | 80.2 | 77.4 | 69.2 | 54.3 | 21.7 | 39.6 | 65.5 | 61.2 | 64.2 | 27.4 | 78.5 | 76.7 |
| Qwen3-VL-Instruct-8B | 8.3B | 0.3B | 8B | 68.3 | 78.4 | 65.0 | 73.6 | 73.7 | 71.4 | 69.3 | 66.2 | 68.5 | 79.1 | 78.3 | 82.2 | 73.4 | 62.5 | 63.1 | 58.4 | 59.9 | 61.9 | 57.9 | 62.0 | 62.6 | 73.8 |
| HunyuanOCR | 1B | 0.4B | 0.6B | 68.3 | 80.2 | 64.3 | 72.4 | 75.0 | 73.1 | 63.0 | 66.1 | 69.9 | 80.3 | 61.4 | 81.9 | 80.6 | 63.7 | 68.3 | 73.1 | 55.6 | 68.9 | 52.2 | 60.7 | 66.8 | 64.2 |
| PaddleOCR-VL | 0.9B | 0.6B | 0.3B | 69.6 | 87.6 | 63.6 | 72.1 | 78.2 | 79.3 | 62.9 | 66.0 | 77.4 | 78.4 | 67.9 | 72.0 | 66.6 | 66.7 | 65.8 | 68.4 | 59.9 | 77.8 | 56.9 | 57.8 | 78.2 | 68.5 |
| olmOCR2 | 7.7B | 0.7B | 7B | 70.4 | 79.9 | 67.2 | 76.7 | 75.7 | 77.3 | 72.5 | 68.9 | 70.6 | 81.0 | 72.0 | 88.0 | 84.0 | 63.3 | 59.0 | 60.8 | 59.4 | 70.6 | 65.8 | 59.2 | 68.6 | 63.4 |
| MinerU-2.5-Pro | 1.2B | 0.7B | 0.5B | 71.0 | 86.2 | 66.1 | 74.6 | 78.3 | 79.5 | 63.4 | 67.4 | 78.0 | 79.7 | 72.1 | 78.6 | 74.2 | 67.0 | 56.6 | 72.2 | 59.1 | 77.6 | 62.6 | 61.8 | 76.5 | 69.7 |
| PaddleOCR-VL-1.6 | 0.9B | 0.6B | 0.3B | 75.0 | 82.8 | 72.6 | 78.0 | 84.1 | 79.7 | 69.2 | 74.8 | 81.6 | 82.0 | 74.7 | 76.4 | 79.3 | 71.6 | 69.4 | 65.6 | 68.7 | 82.5 | 70.7 | 62.3 | 78.0 | 75.7 |
| HunyuanOCR-1.5 | 1B | 0.4B | 0.6B | 76.8 | 86.2 | 73.6 | 79.7 | 79.6 | 80.4 | 74.2 | 70.0 | 81.5 | 84.5 | 78.4 | 86.4 | 82.4 | 73.5 | 71.8 | 71.6 | 65.5 | 75.7 | 67.4 | 77.7 | 80.8 | 77.2 |
| Kimi-K2.5 | 1T | 0.4B | 1T | 77.5 | 85.0 | 75.0 | 81.6 | 85.9 | 86.2 | 72.7 | 71.0 | 80.6 | 86.6 | 77.4 | 87.6 | 86.2 | 72.9 | 75.8 | 74.5 | 72.5 | 70.9 | 61.8 | 67.0 | 81.7 | 78.6 |
| PaddleOCR-VL-1.5 | 0.9B | 0.6B | 0.3B | 78.3 | 87.4 | 75.2 | 81.2 | 84.8 | 83.0 | 75.7 | 78.1 | 83.9 | 85.2 | 80.6 | 80.2 | 78.9 | 74.9 | 71.3 | 67.7 | 69.5 | 86.0 | 76.0 | 68.4 | 84.8 | 75.7 |
| chandra-ocr-2 | 5.3B | 0.5B | 4.8B | 79.7 | 87.8 | 77.1 | 82.7 | 86.6 | 86.5 | 69.7 | 70.3 | 84.6 | 87.4 | 82.7 | 90.7 | 85.6 | 76.4 | 78.2 | 81.1 | 68.8 | 80.3 | 74.0 | 78.5 | 73.8 | 76.3 |
| dots.mocr | 3B | 1.2B | 1.8B | 80.5 | 90.5 | 77.2 | 81.7 | 82.6 | 87.4 | 71.3 | 70.1 | 84.5 | 89.3 | 83.2 | 86.8 | 79.9 | 79.2 | 83.3 | 83.6 | 75.0 | 78.7 | 71.2 | 77.9 | 84.6 | 79.6 |
| MonkeyOCRv2-S-Parsing🤗 | 0.6B | 0.03B | 0.6B | 82.5 | 87.9 | 80.7 | 83.2 | 87.3 | 83.6 | 76.8 | 73.6 | 85.4 | 87.2 | 85.5 | 87.4 | 81.9 | 81.7 | 91.2 | 87.1 | 69.9 | 88.7 | 78.0 | 79.8 | 84.4 | 74.7 |
| MonkeyOCRv2-B-Parsing🤗 | 0.7B | 0.1B | 0.6B | 83.3 | 88.1 | 81.7 | 84.2 | 87.7 | 84.5 | 75.2 | 78.4 | 86.5 | 88.6 | 86.1 | 87.9 | 83.2 | 82.1 | 90.7 | 87.2 | 71.9 | 87.6 | 80.1 | 80.8 | 83.6 | 75.3 |
7. Document understanding performance comparison across different vision foundation models. The evaluation benchmarks are selected following TextMonkey and DT-VQA.
| Model | Params | Overall | DocVQA | InfoVQA | DF | KLC | WTQ | ChartQA | DT-VQA | OCRBench |
|---|---|---|---|---|---|---|---|---|---|---|
| CLIP-B | 86M | 16.0 | 20.1 | 24.2 | 2.3 | 13.8 | 12.8 | 22.2 | 22.3 | 10.6 |
| SigLIP2-B | 93M | 24.9 | 27.0 | 23.5 | 3.1 | 16.7 | 17.4 | 35.0 | 41.5 | 35.1 |
| RADIOv2.5-B | 98M | 37.5 | 60.3 | 31.2 | 29.9 | 30.4 | 29.7 | 51.1 | 44.2 | 23.1 |
| OpenVision-B | 87M | 44.0 | 63.3 | 30.7 | 19.8 | 33.1 | 31.1 | 58.3 | 62.6 | 52.9 |
| DINOv3-B | 86M | 16.1 | 26.5 | 20.8 | 5.6 | 13.2 | 14.0 | 28.9 | 15.8 | 3.9 |
| SAM-B | 90M | 25.2 | 37.8 | 22.2 | 4.7 | 17.5 | 17.6 | 46.5 | 33.3 | 21.9 |
| SAM2-B | 69M | 22.3 | 32.5 | 21.9 | 2.7 | 15.8 | 16.6 | 40.2 | 30.3 | 18.4 |
| oCLIP | 24M | 12.4 | 14.8 | 19.5 | 1.4 | 7.4 | 11.4 | 17.9 | 19.2 | 7.4 |
| DiT | 86M | 8.9 | 11.3 | 20.9 | 0.9 | 5.2 | 9.9 | 12.0 | 9.2 | 1.9 |
| MonkeyOCRv2-S*🤗Link | 28M | 55.9 | 79.3 | 44.5 | 65.1 | 37.6 | 43.0 | 62.0 | 63.1 | 52.2 |
| MonkeyOCRv2-B*🤗Link | 113M | 57.2 | 79.3 | 46.3 | 65.8 | 38.2 | 43.2 | 62.0 | 64.3 | 58.1 |
MonkeyDoc v2
MonkeyDoc v2 is currently the largest document image pre-training image-text pair dataset, comprising 113 million document images across 17 languages. The open-sourcing of MonkeyDoc v2 is still underway. So far, we have released all 52 million synthetic samples and 41 million real-world samples derived from FinePDF. Additional real-world data from other sources is currently being organized and will be released progressively.
Expert Model Labeling Toolchain
We adopt a multi-expert labeling pipeline to obtain reliable annotations for documents. The pipeline includes the following steps:
- Structure Detection
We use dots.mocr for document structure detection and reading-order prediction. The detected regions, including text blocks, tables, formulas, and other layout elements, are cropped from the original page image for subsequent recognition. - Content Recognition
Each cropped block is independently recognized by three expert models: dots.mocr, PaddleOCR-VL, and Qwen3-VL. These complementary models provide multiple annotations for the same block, reducing reliance on any single OCR system. - Block-Level Agreement Filtering
We compare the recognition results from the three expert models and filter out blocks with low agreement. For retained blocks, we select the prediction that has the highest average agreement with the other two predictions as the final block-level annotation. - Page-Level Quality Control
Pages containing any filtered block are discarded. In addition, we use Qwen3 to verify whether the predicted reading order is reasonable, and Qwen3-VL to check whether document regions are missed during structure detection. This multi-expert agreement strategy reduces model-specific annotation errors and improves the reliability of the generated annotations.
References
- dots.mocr: https://github.com/rednote-hilab/dots.mocr
- PaddleOCR-VL: https://github.com/PaddlePaddle/PaddleOCR
- Qwen3-VL: https://github.com/QwenLM/Qwen3-VL
- Qwen3: https://github.com/QwenLM/Qwen3
- MonkeyOCR: https://github.com/Yuliang-Liu/MonkeyOCR
- MDPBench: https://github.com/Yuliang-Liu/MultimodalOCR
- MonkeyDoc: https://modelscope.cn/datasets/zenosai/MonkeyDoc
- FinePDF: https://huggingface.co/datasets/HuggingFaceFW/finepdfs
Citation
If you use any part of this release — the MonkeyOCRv2 encoders, MonkeyOCRv2-Parsing, MonkeyOCRv2-Und, the MDPBench benchmark, or the MonkeyDoc v2 dataset — please cite:
@article{liu2026monkeyocrv2,
title = {MonkeyOCRv2: A Visual-Text Foundation Model for Document AI},
author = {Liu, Yuliang and Li, Zhang and Zhang, Ziyang and Zhang, Shuo and
Liu, Qiang and Song, Jiajun and Guo, Zidun and Wang, Xinhan and
Zheng, Handong and Liu, Yang and Luo, Dongliang and Ma, Zhiyin and
Zhang, Jiarui and Bai, Xiang},
journal = {arXiv preprint arXiv:2607.11562},
year = {2026}
}
Copyright
We warmly welcome your feedback, suggestions, and contributions, which are essential to the continued development and improvement of our framework. The models are are released under the Apache License 2.0 and are free for both research and commercial use. For any questions, please contact us at xbai@hust.edu.cn or ylliu@hust.edu.cn.
- Downloads last month
- 85