we do not have a full checkpoint conversion validation, if you encounter pipeline loading failure and unsidered output, please contact me via bili_sakura@zju.edu.cn

BiliSakura/HSIGene

Hyperspectral image generation — HSIGene converted to diffusers format. Supports task-specific conditioning with local controls (HED, MLSD, sketch, segmentation), global controls (content or text), or metadata embeddings. Outputs 48-band hyperspectral images (256×256 pixels).

Source: HSIGene. Converted to diffusers format; model dir is self-contained (no external project for inference).

Repository Structure (after conversion)

Component	Path
UNet (LocalControlUNet)	unet/
VAE	vae/
Text encoder (CLIP)	text_encoder/
Local adapter	local_adapter/
Global content adapter	global_content_adapter/
Global text adapter	global_text_adapter/
Metadata encoder	metadata_encoder/
Scheduler	scheduler/
Pipeline	pipeline_hsigene.py
Config	model_index.json

Usage

Inference Demo (DiffusionPipeline.from_pretrained)

from diffusers import DiffusionPipeline
pipe = DiffusionPipeline.from_pretrained(
  "/path/to/BiliSakura/HSIGene",
  trust_remote_code=True,
  custom_pipeline="path/to/pipeline_hsigene.py",
  model_path="path/to/BiliSakura/HSIGene"
)
pipe = pipe.to("cuda")

Dependencies: pip install diffusers transformers torch einops safetensors

Per-Condition Inference Demos (Not Combined)

local_conditions shape: (B, 18, H, W); global_conditions shape: (B, 768); metadata shape: (7,) or (B, 7).

# HED condition
output = pipe(prompt="", local_conditions=hed_local, global_conditions=None, metadata=None)

# MLSD condition
output = pipe(prompt="", local_conditions=mlsd_local, global_conditions=None, metadata=None)

# Sketch condition
output = pipe(prompt="", local_conditions=sketch_local, global_conditions=None, metadata=None)

# Segmentation condition
output = pipe(prompt="", local_conditions=seg_local, global_conditions=None, metadata=None)

# Content condition (global)
output = pipe(prompt="", local_conditions=None, global_conditions=content_global, metadata=None)

# Text condition
output = pipe(prompt="Wasteland", local_conditions=None, global_conditions=None, metadata=None)

# Metadata condition
output = pipe(prompt="", local_conditions=None, global_conditions=None, metadata=metadata_vec)

Model Sources

• Paper: HSIGene: A Foundation Model For Hyperspectral Image Generation
• Checkpoint: GoogleDrive
• Annotators: BaiduNetdisk (code: 98f1) → data_prepare/annotator/ckpts

Citation

@article{pangHSIGeneFoundationModel2026,
  title = {{{HSIGene}}: {{A Foundation Model}} for {{Hyperspectral Image Generation}}},
  shorttitle = {{{HSIGene}}},
  author = {Pang, Li and Cao, Xiangyong and Tang, Datao and Xu, Shuang and Bai, Xueru and Zhou, Feng and Meng, Deyu},
  year = 2026,
  month = jan,
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  volume = {48},
  number = {1},
  pages = {730--746},
  issn = {1939-3539},
  doi = {10.1109/TPAMI.2025.3610927},
  urldate = {2026-01-02},
  keywords = {Adaptation models,Computational modeling,Controllable generation,deep learning,diffusion model,Diffusion models,Foundation models,hyperspectral image synthesis,Hyperspectral imaging,Image synthesis,Noise reduction,Reliability,Superresolution,Training}
}