PerX2CT — Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays

This repository hosts the pretrained checkpoints for PerX2CT, a framework that reconstructs a 3D CT volume from two orthogonal (biplanar) 2D X-ray images, using a perspective-projection-based feature sampling scheme.

Paper: Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays (ICASSP 2023, Best Student Paper Award) — https://arxiv.org/abs/2303.05297
Code: https://github.com/dek924/PerX2CT
Authors: Daeun Kyung, Kyungmin Jo, Jaegul Choo, Joonseok Lee, Edward Choi

⚠️ These are weights only. The .ckpt files cannot be used on their own. You must clone the code from the GitHub repository above to load and run them (see Usage).

Checkpoints

File	Paired config (in the GitHub repo)	Notes
`PerX2CT.ckpt`	`configs/PerX2CT.yaml`	Base PerX2CT model (full-frame reconstruction).
`PerX2CT_global.ckpt`	`configs/PerX2CT_global_w_zoomin.yaml`	PerX2CT-global model, trained with zoom-in; supports zoom-in (sub-region) reconstruction.

Each checkpoint must be paired with its matching config; using the wrong config will not load correctly.

Usage

These weights require the original codebase.

# 1. Get the code
git clone https://github.com/dek924/PerX2CT.git
cd PerX2CT

# 2. Environment
conda create -n perx2ct python=3.8
conda activate perx2ct
pip install --upgrade pip
pip install torch==1.8.1+cu111 torchvision==0.9.1+cu111 torchaudio==0.8.1 \
  -f https://download.pytorch.org/whl/torch_stable.html
pip install -r requirement.txt

# 3. Download the checkpoints from this Hugging Face repo, then run inference.

Full-frame evaluation

python main_test.py \
  --ckpt_path /path/to/PerX2CT.ckpt \
  --config_path configs/PerX2CT.yaml \
  --save_dir /path/to/output \
  --val_test test

Zoom-in evaluation (use the PerX2CT_global.ckpt + configs/PerX2CT_global_w_zoomin.yaml)

python main_test_zoom.py \
  --ckpt_path /path/to/PerX2CT_global.ckpt \
  --config_path configs/PerX2CT_global_w_zoomin.yaml \
  --save_dir /path/to/output \
  --zoom_size <target_zoomin_patch_size>

See the GitHub README for the full set of options and for training instructions.

Training data

The model was trained on the LIDC-IDRI chest CT collection, obtained from The Cancer Imaging Archive (TCIA). Synthetic biplanar X-rays were generated from the CT volumes following the projection pipeline introduced in the X2CT-GAN work.

Use of these weights is subject to the data's terms:

LIDC-IDRI is distributed under the Creative Commons Attribution 3.0 Unported (CC BY 3.0) license and the TCIA Data Usage Policy. You must give proper attribution (see Citation).
Per the TCIA terms of use, you must not attempt to re-identify the research participants in the underlying data.

Intended use & limitations

Intended use: research and educational purposes only — method reproduction, benchmarking, and academic study.
Not for clinical use: this model is not a medical device and must not be used for diagnosis, treatment decisions, or any clinical purpose.
Domain limits: trained on LIDC-IDRI chest CT with synthetic X-ray inputs. Performance on real clinical X-rays, other anatomy, other scanners/protocols, or other populations is not guaranteed and is likely to degrade under distribution shift.

License

Weights (the .ckpt files in this repository): Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0). Non-commercial use only; attribution required.
Code: not included here. It lives in the GitHub repository under that project's terms. Note that the data-preprocessing pipeline there is adapted from X2CT-GAN, which is GPLv3, while the model backbone (taming/) is MIT. The inference code needed to run these weights does not require the GPLv3 preprocessing scripts.
Underlying data (LIDC-IDRI): CC BY 3.0 + TCIA Data Usage Policy, as described above.

Citation

If you use these weights, please cite the PerX2CT paper and the LIDC-IDRI / TCIA sources.

PerX2CT

@INPROCEEDINGS{kyung2023perx2ct,
  author={Kyung, Daeun and Jo, Kyungmin and Choo, Jaegul and Lee, Joonseok and Choi, Edward},
  booktitle={ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  title={Perspective Projection-Based 3d CT Reconstruction from Biplanar X-Rays},
  year={2023},
  pages={1-5},
  doi={10.1109/ICASSP49357.2023.10096296}}

LIDC-IDRI (publication)

@article{armato2011lidc,
  title={The Lung Image Database Consortium (LIDC) and Image Database Resource Initiative (IDRI): A completed reference database of lung nodules on CT scans},
  author={Armato III, Samuel G. and McLennan, Geoffrey and Bidaut, Luc and others},
  journal={Medical Physics},
  volume={38}, number={2}, pages={915--931}, year={2011},
  doi={10.1118/1.3528204}}

TCIA

@article{clark2013tcia,
  title={The Cancer Imaging Archive (TCIA): Maintaining and Operating a Public Information Repository},
  author={Clark, Kenneth and Vendt, Bruce and Smith, Kirk and others},
  journal={Journal of Digital Imaging},
  volume={26}, number={6}, pages={1045--1057}, year={2013},
  doi={10.1007/s10278-013-9622-7}}

The LIDC-IDRI data citation / DOI (separate from the publication above) is provided on the current TCIA collection page — please copy it from https://www.cancerimagingarchive.net/collection/lidc-idri/ to ensure it is up to date.

Acknowledgements

This work builds on:

CompVis/taming-transformers (MIT) — VQGAN backbone.
kylekma/X2CT (GPLv3) — CT-to-X-ray projection method used for data preprocessing.

Downloads last month: -; Downloads are not tracked for this model. How to track

Paper for KAISTEdlab/PerX2CT

Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays

Paper • 2303.05297 • Published Mar 9, 2023