Hugging Face's logo

Scicom-intl
/
WideCodec

Audio-to-Audio
PyTorch
neucodec
neural-audio-codec
widecodec
44100hz
decoder-finetune
speech
Model card Files
xet
 Community

WideCodec β€” 44.1 kHz decoder

A SOTA 0.8 kbps, 44.1 kHz audio tokenizer.

WideCodec is a 44.1 kHz decoder finetune of neuphonic/neucodec.

  • Sample rate: 44 100 Hz Β· hop: 882 Β· 50 tokens/sec

Inference (self-contained)

Everything needed to run inference is in this repo β€” the neucodec/ package and infer_widecodec.py. No other source code required.

pip install torch transformers huggingface_hub local-attention einops librosa soundfile
huggingface-cli login          # if this repo is private (or export HF_TOKEN=hf_...)

huggingface-cli download Scicom-intl/WideCodec --local-dir WideCodec
cd WideCodec
python infer_widecodec.py --input my.wav --out-dir out      # one file
python infer_widecodec.py --input folder/  --out-dir out    # a directory

Or load it directly in Python:

import sys; sys.path.insert(0, "WideCodec")   # the downloaded repo dir
import torch, librosa
from neucodec import NeuCodec

model = NeuCodec._from_pretrained(model_id="Scicom-intl/WideCodec", decoder_depth=20).eval().cuda()
wav16, _ = librosa.load("my.wav", sr=16000, mono=True)   # encoder ingests 16 kHz mono
x = torch.from_numpy(wav16).float().view(1, 1, -1).cuda()
with torch.no_grad():
    codes  = model.encode_code(x)      # frozen FSQ codes β€” 0.8 kbps, 50 tok/s
    wav44  = model.decode_code(codes)  # 44.1 kHz reconstruction

Files

file what
pytorch_model.bin decoder weights for inference (load with NeuCodec._from_pretrained(..., decoder_depth=20))
last.ckpt full PyTorch-Lightning checkpoint with optimizer states + LR schedulers for resuming training

How WideCodec compares (44 kHz-class, by bitrate)

Offline benchmark on a heavily multilingual 9,291-clip wideband set (50 clips Γ— 188 clean β‰₯44.1 kHz datasets, 400+ language/dialect tags), faster-UTMOSv2 + spectral vs ground truth.

bitrate vs UTMOSv2

WideCodec release progression

codec native SR tokens/s codebooks ~bitrate UTMOSv2 mel-L1 ↓ HFβ‰₯11k rolloff
ground truth β€” β€” β€” β€” 2.822 β€” 0.00244 8263
nvidia nemo44k 44.1k ~86 RVQ (many) ~6–9 kbps 2.903 0.379 0.00196 8372
WideCodec (ours, 2026-08-16) 44.1k 50 1 ~0.8 kbps 2.919 0.571 0.00223 9011
WideCodec (previous, 2026-07-21) 44.1k 50 1 ~0.8 kbps 2.788 0.571 0.00223 9011
dac 44.1k 86 9 ~8 kbps 2.672 0.341 0.00156 8185
snac44k 44.1k multi-scale 3–4 ~2.6 kbps 2.340 0.493 0.00158 8402
encodec48k 48k 150 RVQ 24 kbps 2.042 0.458 0.00169 8800

Bitrate = frame-rate Γ— Ξ£ logβ‚‚(codebook size). WideCodec's FSQ is levels=[4]Γ—8, num_quantizers=1 β†’ 8Β·logβ‚‚4 = 16 bits/frame Γ— 50 frames/s = 800 bps = 0.8 kbps (vs DAC's 9 codebooks Γ— 10 bits Γ— 86 fps β‰ˆ 8 kbps).

The 2026-08-16 release reaches UTMOSv2 2.919 on the 9,291-clip set β€” above ground truth (2.822) and level with NVIDIA NeMo-44k (2.903), at ~1/10 of NeMo's bitrate. It gained +0.131 UTMOSv2 over the previous release at unchanged content accuracy: CV22 macro CER is 7.70% for both (micro 7.09%), well inside the measured run-to-run spread of Β±0.25 pt.

Same architecture, same losses, same 50 tokens/s and single codebook β€” only the training corpus changed, to clean DNSMOS-gated β‰₯44.1 kHz audio. The gain is not an artifact of the benchmark pool: on a held-out set of 1,330 clips from 138 sources absent from training, it scores 2.807 vs the previous release's 2.654 (+0.153, paired t=+14.2), and reference-based PESQ-WB also improves (+0.017, t=+7.5) β€” so naturalness and fidelity moved together rather than trading off. Full report: evaluation/REPORT_44k_class_codecs.md.

Training stages / which revision to pull

This model is trained in two stages. main always points at the latest stage.

stage data how to load
Stage 1 β€” noisy mix (steps 0β†’1.58M, epoch=9-step=1580000) 8 base corpora + scale44k (~6,500 h, mostly noisy crowdsourced/podcast) pin revision d2c612f4319c1c1e0c65a9431a7d6db1b7955ce0
Stage 2 β€” clean finetune (from 1.58M, clean β‰₯44.1 kHz only; in progress) TTS-Clean44k + Clean-Podcast + clean-teacher pool + EARS/Expresso main (default) 
# the `neucodec/` package is bundled in this repo (see 'Inference (self-contained)')
from neucodec import NeuCodec

# Stage 1 (first-stage noisy training) β€” pin the exact revision:
model = NeuCodec._from_pretrained(
    model_id="Scicom-intl/WideCodec",
    decoder_depth=20,
    revision="d2c612f4319c1c1e0c65a9431a7d6db1b7955ce0",
)

Training data

Diverse, predominantly high-sample-rate human speech (no synthetic TTS). The scale-up group is SR-verified β‰₯ 44 kHz (probed before inclusion); the base corpora are long-form conversational, expressive and anechoic speech. Two groups:

Base corpora (Malaysia-AI + expressive/anechoic speech)

dataset content
malaysia-ai/malaysian-podcast-youtube Malay long-form conversational podcasts (~2,234 h)
malaysia-ai/singaporean-podcast-youtube Singaporean English podcasts (~1,255 h)
malaysia-ai/Multilingual-TTS CommonVoice-sidon multilingual short clips
malaysia-ai/malaysian-cartoons-youtube 48 kHz cartoon dialogue (subset)
malaysia-ai/malaysian-movie-youtube 48 kHz Malay movie dialogue
ylacombe/expresso 48 kHz expressive read speech
nytopop/expresso-conversational 48 kHz improvised dialogue
EARS 48 kHz anechoic fullband (emotional / conversational / reading)

Scale-up: 336 verified β‰₯ 44 kHz datasets

To maximise acoustic diversity for the decoder (language-agnostic β€” the codebook is frozen), training is scaled with 336 additional datasets (162 at 48 kHz, 174 at 44.1 kHz), mirrored as <name>*audio.zip in malaysia-ai/Multilingual-TTS and SR-verified β‰₯ 44 kHz. Original sources:

Full list of 336 β‰₯ 44 kHz source datasets

Usage

See the canonical Load from Hugging Face section in the source repo. decoder_depth=20 is required β€” the weights are a depth-20 decoder, so loading with any other depth mismatches the architecture.

import soundfile as sf

# the `neucodec/` package is bundled in this repo (see 'Inference (self-contained)')
from neucodec import NeuCodec

# decoder_depth=20 MUST match this repo; pass token=... (or hf login) for access
model = NeuCodec._from_pretrained(model_id="Scicom-intl/WideCodec", decoder_depth=20)
model = model.eval().cuda()

# encode (16 kHz path, frozen) -> codes -> decode (44.1 kHz, this finetune)
codes = model.encode_code("input.wav")                   # [1, 1, T], identical to base NeuCodec
recon_44k = model.decode_code(codes).squeeze().cpu().numpy()
sf.write("recon.wav", recon_44k, model.sample_rate)      # 44100

The discrete codes are identical to base neuphonic/neucodec β€” only the reconstruction sample rate and fidelity differ.

Downloads last month

-

Downloads are not tracked for this model. How to track
Inference Providers NEW
Audio-to-Audio
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Collection including Scicom-intl/WideCodec