ai-toolkit/melbandroformer

Mel-Band RoFormer (vocals) for ai-toolkit

Safetensors repack of Kimberley Jensen's Mel-Band RoFormer vocal separation model, used by ai-toolkit to split songs into a vocals track and an instrumental track (instrumental = mix - vocals).

These weights are not ours. All credit goes to the original authors:

• Weights: KimberleyJSN/melbandroformer (MelBandRoformer.ckpt, MIT) by Kimberley Jensen
• Training framework and config: ZFTurbo/Music-Source-Separation-Training (config_vocals_mel_band_roformer_kj.yaml) by Roman Solovyev
• Architecture implementation: lucidrains/BS-RoFormer by Phil Wang
• Paper: Mel-Band RoFormer for Music Source Separation, Wang et al. 2023

Files

file	notes
melbandroformer_vocals_kj.safetensors	fp32, byte-identical tensors to the original .ckpt. Model kwargs and inference defaults (chunk_size, num_overlap) are stored in the safetensors metadata, so no separate config is needed.

Conversion script: scripts/convert_melbandroformer.py.

Usage

ai-toolkit downloads this file automatically on first use:

python -m toolkit.audio.melbandroformer song.flac
# -> song_vocals.flac, song_instrumental.flac

from toolkit.audio.melbandroformer import load_melbandroformer, separate
model = load_melbandroformer(device="cuda", compile=True)
vocals, instrumental = separate(model, wav, sample_rate)  # wav: [channels, samples]

Input: mono or stereo at any sample rate (resampled to 44.1 kHz internally, output at the input rate).

License

MIT, same as the original weights and code.