AnyCam-MCT — Multi-Frame Calibration Transformer

Final checkpoint of the master's thesis "Learning Camera Geometry from Unlabeled Real-World Dynamic Video" (TU Munich, 2026): AnyCam (CVPR 2025) extended with the Multi-Frame Calibration Transformer (MCT), a ~25M-parameter cross-frame attention module that aggregates AnyCalib features over multiple frames for video calibration, coupled to AnyCam's pose head via a focal embedding. Trained fully self-supervised on ~82k frames of in-the-wild video; this checkpoint includes the August 2026 correction pass (evaluation bugs fixed, calibration branch retrained with proper input normalization, checkpoint selected by validation loss).

Code, benchmarks, and reproduction instructions: https://github.com/kalman17/mcvo

Headline results (honest protocol, raw rows in the GitHub repo; tables updated 17 August 2026 — see the repo CHANGELOG)

Measurement	This model	Competitors measured
Focal error, native wide KITTI frames (own preprocessing each)	15.7 %	11.6 % VGGT-1B · 14.2 % AnyCalib · 20.4 % Pi3 · 38.0 % DA3
Focal error, KITTI windows	20.4 %	18.4 % AnyCalib · 66.9 % AnyCam
Focal error, Sintel / TUM-RGBD windows	20.7 % / 12.9 %	20.1 % / 11.2 % AnyCalib
Rotation error, Sintel (median)	0.40°	0.50° (AnyCam)

Known limitations (stated in the repo README): calibration is on par with the single-image specialist, not beyond it, and multi-frame aggregation does not beat per-frame averaging for this checkpoint; translation direction on TUM-RGBD is worse than AnyCam; large supervised models (Depth Anything 3) lead absolute pose accuracy; full-trajectory drift vs long-context AnyCam inference.

Cost (NVIDIA A40, 4-frame window, end-to-end)

	Params	Peak GPU mem	Latency
MCT calibration branch alone	346 M (25 M trained)	1.4 GiB	161 ms
Full thesis pipeline (MCT + AnyCam pose, incl. depth/flow nets)	460 M	5.0 GiB	820 ms
Follow-up image-only VO model (thekman17/mcvo)	154 M	0.69 GiB	75 ms

Same GPU / windows / protocol as the comparison against VGGT, π³ and Depth Anything 3 in the GitHub README (honest_benchmarks/latency_summary.json).

Files

• mct_final.pt — merged inference checkpoint (pose predictor + MCT calibration branch; load with experiments/benchmark_phase_c_checkpoints.py:create_inference_model
  • load_phase_c_checkpoint from the GitHub repo, input_normalization=True).

License / attribution

Apache-2.0. Contains weights derived from AnyCam (MIT, Wimbauer et al., CVPR 2025) and AnyCalib (Apache-2.0, Tirado-Garín et al., ICCV 2025); DINOv2 backbones (Apache-2.0, Meta). Please cite those works alongside the thesis.