SegviGen: Repurposing 3D Generative Model for Part Segmentation

SegviGen is a framework for 3D part segmentation that leverages the rich 3D structural and textural knowledge encoded in large-scale 3D generative models. It learns to predict part-indicative colors while reconstructing geometry, and unifies three settings in one architecture: interactive part segmentation, full segmentation, and 2D segmentation map–guided full segmentation with arbitrary granularity.

• Paper: SegviGen: Repurposing 3D Generative Model for Part Segmentation
• Project Page: https://fenghora.github.io/SegviGen-Page/
• Repository: https://github.com/Nelipot-Lee/SegviGen
• Demo: Hugging Face Space

Features

• Repurposed 3D Generative Priors for Data Efficiency: Reuses knowledge from native 3D generative models, requiring only 0.32% training data.
• Unified and Flexible Settings: Supports interactive, full, and 2D-guided segmentation in a single architecture.
• State-of-the-Art Accuracy: Delivers significant improvements in IoU over existing methods like P3-SAM.

Usage

For installation, please refer to the official GitHub repository.

Interactive part-segmentation

python inference_interactive.py \
    --ckpt_path path/to/interactive_seg.ckpt \
    --glb ./data_toolkit/assets/example.glb \
    --input_vxz ./data_toolkit/assets/input.vxz \
    --transforms ./data_toolkit/transforms.json \
    --img ./data_toolkit/assets/img.png \
    --export_glb ./data_toolkit/assets/output.glb \
    --input_vxz_points 388 448 392

Full segmentation

python inference_full.py \
    --ckpt_path path/to/full_seg.ckpt \
    --glb ./data_toolkit/assets/example.glb \
    --input_vxz ./data_toolkit/assets/input.vxz \
    --transforms ./data_toolkit/transforms.json \
    --img ./data_toolkit/assets/img.png \
    --export_glb ./data_toolkit/assets/output.glb

Full segmentation with 2D guidance

python inference_full.py \
    --ckpt_path path/to/full_seg_w_2d_map.ckpt \
    --glb ./data_toolkit/assets/example.glb \
    --input_vxz ./data_toolkit/assets/input.vxz \
    --img ./data_toolkit/assets/full_seg_w_2d_map/2d_map.png \
    --export_glb ./data_toolkit/assets/output.glb \
    --two_d_map

Citation

@article{li2026segvigen,
      title = {SegviGen: Repurposing 3D Generative Model for Part Segmentation}, 
      author = {Lin Li and Haoran Feng and Zehuan Huang and Haohua Chen and Wenbo Nie and Shaohua Hou and Keqing Fan and Pan Hu and Sheng Wang and Buyu Li and Lu Sheng},
      journal = {arXiv preprint arXiv:2603.16869},
      year = {2026}
}