OracleZoom

Shubhashis Roy Dipta*, Sourajit Saha*, Shaswati Saha, Nobin Sarwar · University of Maryland, Baltimore County

*Equal contribution.

Reference-constrained recursive super-resolution, inspired by on-policy self-distillation.
OracleZoom drives recursive 4x super-resolution out to 256x while staying faithful to the observed scene. Ground truth runs out at 4x, so OracleZoom trains on its own recursive predictions and uses the last available ground-truth image as a reference past that boundary. This repo is self-contained: the merged model, the inference code, and the required checkpoints are all here. You only download two public base models (Stable Diffusion 3-medium, Qwen2.5-VL-3B) automatically.

Try it in the browser: 🤗 Space (no install, runs on ZeroGPU)

Paper: arXiv:2609.06490 (HF paper page) · Code: https://github.com/dipta007/OracleZoom · Project page: https://dipta007.github.io/OracleZoom/ · Everything in one place: 🤗 collection

Quickstart (one image, all scales)

No GPU? Use the 🤗 Space instead. To run it yourself you need one NVIDIA GPU (~16 GB) and Python 3.10.

# 0. One-time: Stable Diffusion 3 is gated, so accept its license on HF, then log in
pip install -U "huggingface_hub[cli]"
hf auth login

# 1. Download this repo (merged model + code + checkpoints)
hf download dipta007/OracleZoom --local-dir OracleZoom
cd OracleZoom

# 2. Install dependencies
pip install -r requirements.txt

# 3. Super-resolve ONE image (4x -> 16x -> 64x -> 256x)
python inference.py --input /path/to/photo.jpg --output ./outputs

Results in ./outputs/: photo_1x.png (the 512x512 input crop), photo_4x.png, photo_16x.png, photo_64x.png, photo_256x.png.

Stable Diffusion 3-medium and Qwen2.5-VL-3B download automatically on first run.

Batching many images: inference.py exposes zoom_image(sr, model, proc, pvi, image_path, out_dir). Build the models once (build_sr(...), build_vlm(...)) and call zoom_image in a loop over your images.

Training data

The curated training set is released separately at dipta007/OracleZoom-4KLSDB-train. The released model uses its 1k config (1,000 curated 4K images).

What's in this repo

Path	What it is
merged_transformer.safetensors	The OracleZoom super-resolution transformer (SD3 + Chain-of-Zoom's SR module + our distilled adapter, merged), fp32, ~8.35 GB.
inference.py	Self-contained runner: one image in, all scales out (recursive zoom + VLM prompting).
coz/	Vendored Chain-of-Zoom inference code (the one-step SR wrapper + helpers).
ckpt/	Chain-of-Zoom's SR-VAE and VLM-prompt (Qwen LoRA) checkpoints needed by the pipeline.
requirements.txt	Python dependencies.

Method

Recursive SR (Chain-of-Zoom) reuses a 4x backbone step after step to reach 16x-256x. The ground truth needed to supervise those steps grows geometrically: a 256x target would need 52 GB per image. So supervision stops at 4x, and every deeper step is blind, seeing only a blurred crop of its own previous output. Errors compound and the invented detail may be hallucinated.

Training on the model's own predictions, constrained by the last reference. Training follows the model's own 4x then 16x predictions and backpropagates through both steps. Five objectives separate what the reference can verify from what it cannot:

Objective	What it does	Weight
Direct supervision	LPIPS between the decoded 4x prediction and ground truth.	w_4x 1.0
Cross-scale consistency	Project the 16x prediction back to the reference resolution, then match the aligned region of the 4x ground truth.	w_deep 1.0
Quality guidance	Frozen TOPIQ-NR guides the detail the reference cannot verify.	beta_reward 0.4
KL prior	Keep adapted latents close to the pretrained model's prediction on the same input.	beta_kl 8.0
EMA consistency	A slowly updated adapter copy (decay 0.95), run on the ground-truth input, stabilizes training at the supervision boundary.	lambda_ema 0.1

Only a rank-16 adapter is trained (7.1M parameters, 1,000 curated 4K images); the backbone, VAE, and prompter stay frozen. The KL prior is what keeps quality guidance honest: removing it raises 16x CLIPIQA from 0.714 to 0.794, but worsens projected DISTS from 0.215 to 0.330 and raises judged hallucination from 0.303 to 0.907. The released weights have the adapter already merged in.

Results

Every method runs inside the same zoom loop and is scored by the same code, over seven test sets (4KLSDB, DIV2K, DIV8K, DRealSR, FFHQ, Flickr2K, RealSR) and 4x to 256x.

Axis	Metric	Ours	Best baseline
Quality (no-reference)	CLIPIQA, mean over scales	0.713	0.621 (Chain-of-Zoom)
Quality (no-reference)	CLIPIQA @256x	0.706	0.579 (Chain-of-Zoom)
Fidelity @4x (ground truth exists)	LPIPS	0.199	0.215 (Chain-of-Zoom)
Fidelity @4x	DISTS	0.160	0.164 (SeeSR)
Deeper scales (InternVL3.5-38B judge)	preferred over Chain-of-Zoom	68% @64x, 78% @256x	-
Deeper scales	hallucination rate	0.21 @64x, 0.14 @256x	0.55, 0.70 (Chain-of-Zoom)

Past 4x there is no ground truth, so the judge measures consistency with the preceding zooms, not recovery of unseen detail. Ties and abstentions are excluded from the win rate. No-reference quality scores alone do not establish agreement with the observed scene.

Intended Use

• In-scope: research on faithful extreme (recursive) super-resolution of natural photographs.
• Out-of-scope: forensic/evidentiary use (detail past 4x is generated, not recovered); real-camera-zoom claims (the benchmark uses synthetic center-crop zoom).

Acknowledgements & Licensing

The coz/ code and the checkpoints in ckpt/ are from Chain-of-Zoom and are redistributed here for convenience; please respect their original license and cite them. The pipeline uses Stable Diffusion 3-medium and Qwen2.5-VL-3B under their respective licenses. OracleZoom's own contribution (the trained adapter, merged into merged_transformer.safetensors) is released for research, non-commercial use (CC-BY-NC-4.0).

Citation

@misc{dipta2026oraclezoomonpolicyselfdistillationinspired,
  title={OracleZoom: On-Policy Self-Distillation Inspired Reference-Constrained Recursive Image Super Resolution},
  author={Shubhashis Roy Dipta and Sourajit Saha and Shaswati Saha and Nobin Sarwar},
  year={2026},
  eprint={2609.06490},
  archivePrefix={arXiv},
  primaryClass={cs.CV},
  url={https://arxiv.org/abs/2609.06490}
}

Please also cite Chain-of-Zoom and OSEDiff.