all_sft_formats_unbalanced_20251122_ep3_lr3e5_qwen3-vl-8b

This model is a fine-tuned version of Qwen/Qwen3-VL-8B-Instruct on the all_sft_formats_unbalanced_20251122_part_1 dataset.

It was developed as part of the research paper: "Building a Precise Video Language with Human-AI Oversight" (CVPR 2026 Highlight).

Paper: Building a Precise Video Language with Human-AI Oversight
Project Page: https://linzhiqiu.github.io/papers/chai/
GitHub Repository: https://github.com/chancharikmitra/CHAI

Model description

This model belongs to a family of video-language models (VLMs) optimized for precise video captioning using the CHAI (Critique-based Human–AI Oversight) framework. CHAI pairs trained human experts with model-generated pre-captions: experts provide correctional critiques that guide revisions into improved post-captions.

Intended uses & limitations

This model is intended for research in:

Precise video captioning and cinematography-aware description.
Multimodal reward modeling and binary alignment scoring.
Critique generation for video-language tasks.

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 3e-05
train_batch_size: 10
eval_batch_size: 8
seed: 42
distributed_type: multi-GPU
num_devices: 64
gradient_accumulation_steps: 2
total_train_batch_size: 1280
total_eval_batch_size: 512
optimizer: Use adamw_torch_fused with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: cosine
lr_scheduler_warmup_ratio: 0.05
num_epochs: 3.0

Framework versions

Transformers 4.57.1
Pytorch 2.9.1+cu128
Datasets 4.0.0
Tokenizers 0.22.1

Citation

If you find this work useful, please cite:

@inproceedings{chai2026,
  title     = {Building a Precise Video Language with Human--AI Oversight},
  author    = {Zhiqiu Lin and Chancharik Mitra and Siyuan Cen and Isaac Li and Yuhan Huang and Yu Tong Tiffany Ling and Hewei Wang and Irene Pi and Shihang Zhu and Ryan Rao and George Liu and Jiaxi Li and Ruojin Li and Yili Han and Yilun Du and Deva Ramanan},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year      = {2026}
}