NERVE

Norm-free Efficient Restoration for Various Edge devices

A lightweight super-resolution architecture that is simple to train and fast to run. Released models are verified with traiNNer-redux (training) and ONNX Runtime + ncnn (inference). The op set is standard, so other runtimes (CoreML, TensorRT, DirectML, ...) are expected to work too — but they are not tested here.

Part of the BODY suite by Philip Hofmann — networks built to be real-world usable first, not benchmark-chasing. See Relation to HEART for which one to pick.

Short version: copy nerve_arch.py into traiNNer-redux, train with the sample configs, then run the .safetensors, .onnx, or .ncnn models. One architecture, one size, no fused/unfused pairs. Details below.

Input (as-is) | bicubic x4 | NERVE 4x release | NERVE 4x OTF GAN.

Why NERVE?

There are strong and established SR networks already. This is the honest case for when NERVE is worth your time — and when it is not.

This project comes out of a trainer's frustration rather than a benchmark goal. After training hundreds of SISR models (see Phhofm/models), the recurring annoyance was networks that felt like paper networks: they chase leaderboard metrics with techniques that leave you juggling an unfused and a fused checkpoint, or whose dynamic-ONNX conversion is fragile. This network is the opposite of that — built so a trainer can pick it up, train it, and convert it to dynamic ONNX without headaches.

Strengths of NERVE

Real-world usability over metric chasing. Trained on a CC0 dataset with the full Real-ESRGAN OTF degradation pipeline; the OTF GAN model restores degraded images, not just clean bicubic downsamples.
One file, one variant. A single readable architecture file and one model size. No S/M/L matrix, no fused-vs-unfused checkpoints, nothing to fuse before deploy — what you train is what you ship.
Export just works. Dynamic-shape ONNX (verified, ORT parity ~1e-5) and a clean ncnn conversion. Standard ops only (Conv / Add / ReLU / DepthToSpace / Resize).
Boring, stable training. Norm-free, bf16 AMP, no gradient clipping, a single Charbonnier loss for the base stage.
Small and fast. ~1.8M params — SPAN-class size but conv-only, suited to mobile, edge, web and game engines.
Maintainable for years. One op set, no exotic dependencies, ~85-line arch.

When to pick something else. If the goal is maximum PSNR/LPIPS on academic benchmarks, or a large transformer, NERVE is not aiming there — use HEART, HAT, ATD, DAT2, or one of the community networks below. NERVE aims at simplicity, deployability and hassle-free training.

Credit where it is due. None of this exists in a vacuum. SISR has many strong architectures and generous community members, for example HAT, ATD and DAT2, the community network work of umzi and Kim2091, fdat by sharekhan, and the-database (traiNNer-redux, its benchmarks and its models). This list is far from complete — apologies to anyone not named; the SISR community has a great many people contributing a great deal.

Quickstart

As a user (just want to upscale / restore images)

Pick a model and run it:

I want to...	Use
upscale a clean image 4x	`models/4x_NERVE_release.safetensors`
upscale a clean image 2x	`models/2x_NERVE_release.safetensors`
restore a blurry / noisy / JPEG-y image 4x	`models/4x_NERVE_OTF_gan.safetensors`
run on-device / inside an engine	the `onnx/` or `ncnn/` files

ONNX Runtime (verified): use onnx/*.onnx (dynamic H/W). TensorRT / DirectML should work too (same op set) but are untested here.

ncnn (verified, fp16): ncnn/*.ncnn.param + *.ncnn.bin. Feed images with Mat.from_pixels:

import ncnn
net = ncnn.Net(); net.load_param("4x_NERVE_fp16.ncnn.param"); net.load_model("4x_NERVE_fp16.ncnn.bin")
ex = net.create_extractor()
ex.input("in0", ncnn.Mat.from_pixels(bgr_uint8, ncnn.Mat.PixelType.PIXEL_BGR2RGB, w, h))
out = ex.extract("out0")[1]   # CHW float, 4x

(Use from_pixels; the ncnn Python binding's raw-array constructor misbehaves on large inputs. Outputs are fp16, so expect small deviations at hard edges.)

chaiNNer: load the onnx/*.onnx files directly. Loading the .safetensors via spandrel needs NERVE registered in your chaiNNer/spandrel build (not upstream yet).
Input: RGB, any size. Output: 2x or 4x.

As a trainer (want to train your own)

NERVE is a normal traiNNer-redux architecture — no special steps.

# 1. Add the architecture + its ICNR helper (auto-registers: traiNNer scans *_arch.py)
cp nerve_arch.py <traiNNer-redux>/traiNNer/archs/
cp icnr.py       <traiNNer-redux>/traiNNer/utils/   # ICNR init helper (not in upstream yet)

# 2. Copy a training config and point it at your data
cp configs/4x_NERVE_release.yml <traiNNer-redux>/options/
#    -> edit dataroot_gt / dataroot_lq in that file

# 3. Train
cd <traiNNer-redux>
python train.py -opt options/4x_NERVE_release.yml --auto_resume

# 4. Export to optimized dynamic ONNX (official converter)
cp configs/4x_NERVE_onnx.yml <traiNNer-redux>/options/
#    -> set path.pretrain_network_g to your trained checkpoint
python convert_to_onnx.py -opt options/4x_NERVE_onnx.yml

That is the whole workflow — 2 small files plus a config. The net is one file, the config is one file, the checkpoint is one file — no fusion step, no paired fused/unfused weights. (icnr.py is the only piece not in upstream traiNNer-redux yet; it is the ICNR initializer for the PixelShuffle head.)

4x_NERVE_onnx.yml exports dynamic-shape ONNX (opset 20) and can emit fp32, fp16, or bf16 (onnx.dtype), with OnnxSlim optimization and a built-in PyTorch-vs-ONNX verify pass.
Prefer no extra dependencies? scripts/export_dynamic.py is a tiny standalone exporter (torch + onnxruntime only).

Start from a released pretrain (skip the expensive early training) by setting path.pretrain_network_g in your training config to a file from models/:

training stage	warm-start from
`4x_NERVE_release.yml`	`models/4x_NERVE_release.safetensors` (or the 2x one with `strict_load_g: false`)
`4x_NERVE_OTF_fidelity.yml`	`models/4x_NERVE_release.safetensors`
`4x_NERVE_OTF_gan.yml`	your OTF fidelity checkpoint

Training tips (all NERVE models follow these):

bf16 AMP is fine; no gradient clipping needed (norm-free net).
--auto_resume is recommended for long runs.
For real-world input, follow the chain: clean release -> OTF fidelity -> OTF GAN.

Models

File	Role	Notes
`models/2x_NERVE_release.safetensors`	2x official pretrain	clean bicubic, 31.84 dB / 0.9255 SSIM (Urban100)
`models/4x_NERVE_release.safetensors`	4x official pretrain	clean bicubic, 25.09 dB / 0.7531 SSIM (Urban100)
`models/4x_NERVE_OTF_fidelity.safetensors`	4x OTF fidelity	restore degraded input; the pretrain for OTF GAN finetunes
`models/4x_NERVE_OTF_gan.safetensors`	4x OTF GAN	sharpest real-world output (topiq 0.430 / lpips 0.290, Urban100 OTF val)
`onnx/4x_NERVE_1x3xHxW_fp32_op20.onnx`	4x release, dynamic ONNX	onnxslim-optimized, fp32
`onnx/2x_NERVE_1x3xHxW_fp32_op20.onnx`	2x release, dynamic ONNX	onnxslim-optimized, fp32
`onnx/4x_NERVE_OTF_fidelity_1x3xHxW_fp32_op20.onnx`	4x OTF fidelity, dynamic ONNX	onnxslim-optimized, fp32
`onnx/4x_NERVE_OTF_gan_1x3xHxW_fp32_op20.onnx`	4x OTF GAN, dynamic ONNX	onnxslim-optimized, fp32
`ncnn/4x_NERVE_fp16.ncnn.param` + `.bin`	4x release, ncnn	fp16 weights
`ncnn/2x_NERVE_fp16.ncnn.param` + `.bin`	2x release, ncnn	fp16 weights
`ncnn/4x_NERVE_OTF_fidelity_fp16.ncnn.param` + `.bin`	4x OTF fidelity, ncnn	fp16 weights
`ncnn/4x_NERVE_OTF_gan_fp16.ncnn.param` + `.bin`	4x OTF GAN, ncnn	fp16 weights

models/*.safetensors are plain EMA state dicts (no wrapper) — load them with nerve_arch.nerve(scale=N).

All released models (2x/4x release, 4x OTF fidelity, 4x OTF GAN) are final.

Pretrain chain (how the models relate):

2x_NERVE_release ──→ 4x_NERVE_release ──→ OTF fidelity ──→ OTF GAN

The release models are the shared starting point for anyone training NERVE: warm-start from them (2x → 4x, or release → OTF fidelity → OTF GAN) to skip the expensive early training.

Examples

Real images are fed in as-is (not downscaled) and upscaled 4x. For each image there are two comparisons:

examples/*_compare.png — whole image at full resolution: input | bicubic x4 | NERVE 4x release | NERVE 4x OTF GAN.
examples/*_1to1.png — a 1:1 native-pixel center crop.

The bicubic column is the baseline; the OTF GAN column is the released showcase model. The source inputs are in examples/inputs/.

examples/realphoto_compare.png (street scene)
examples/ani_compare.png (anime)
examples/manga_compare.png (manga)
examples/text_compare.png (text)
examples/noisy_compare.png (noisy input)
examples/stablediffusion_compare.png (AI-generated)

Architecture

5x5 conv stem → 24 × (Conv-ReLU-Conv + residual) → PixelShuffle → + bicubic(x)

~1.8M params, pure convolution — no attention, no normalization, no gating, no positional encoding, no reparameterization.
A bicubic input residual means the net only learns what bicubic upscaling misses — that's why it stays small and trains fast.
ICNR-initialized PixelShuffle head (checkerboard-free by construction).
The op graph is trivial: Conv, Add, ReLU, DepthToSpace, Resize. That's the entire reason it exports to dynamic ONNX and converts to ncnn (verified) without custom ops or fused/unfused pairs; CoreML is expected but untested.

What we actually tested

Every design choice here came from a measured ablation, not a guess. The full development log — including dead ends and raw numbers — is in docs/ABLATIONS.md. Highlights:

Normalization: removing LayerNorm beat the normed baseline by +1.15 dB at ~2x the training speed -> norm-free.
Depth/width sweep + capacity scaling: quality improved with size; the sweep put the GAN quality knee at 1.80M (0.69M -> 1.80M: lpips -0.015; 1.80M -> 2.39M: only -0.002 more).
Dilation: a dilated-conv A/B under the OTF recipe was null -> plain convs.
Checkerboard: diagnosed a PixelShuffle phase-lock, fixed with ICNR plus keeping aliased (nearest-exact) LR. An anti-aliased-LR + sub-pixel-jitter variant made thin lines worse and was reverted.
OTF/GAN capability: the small net trained productively under the full Real-ESRGAN OTF GAN recipe (no collapse).
Runtime validation: every released .safetensors loads (strict) and runs; every .onnx loads in ONNX Runtime; the ncnn models were run at multiple input sizes up to 720x720 (2880x2880 output).
Deployability: dynamic-ONNX export + ORT parity and ncnn conversion were verified at every stage (the numbers are in the log).
Not tested here (community welcome): TensorRT / DirectML speed and fixed-shape static exports, CoreML, and on-device benchmarks. Train more variants, benchmark, and share — the arch is standard ops only.

Training data & license

All released NERVE models were trained only on Phips/lucid-cc0-v2-hc-512, a CC0 dataset — the released weights are clean to use commercially.

We did not optimize for benchmark leaderboards. If you want higher PSNR/LPIPS numbers, training on larger academic datasets (DIV2K, Flickr2K, LSDIR, ...) is expected to score better; NERVE is a normal architecture, train it on whatever you like.

Relation to HEART

NERVE and HEART are the two members of the BODY suite (by Philip Hofmann): same philosophy (real-world usability > benchmark scores), different tiers.

	HEART	NERVE
size	~16.7M params	~1.8M params
design	attention (FlashAttention-friendly)	pure convolution
use when	you want the best quality and have the compute (desktop, server, high-end device)	you want speed, small size, and dead-simple deployment (mobile, edge, web, game engines)
cost	heavier to run	a fraction of the cost

Rule of thumb: HEART for maximum quality, NERVE for maximum practicality. Both export to dynamic ONNX; NERVE additionally converts to ncnn cleanly because it has no attention ops.

Side-by-side comparisons on the same inputs (input | bicubic | both release models | both OTF GAN models), each with a 1:1 crop: examples/OST_009_nerve_vs_heart_* and examples/00003_nerve_vs_heart_* (Real-ESRGAN test set), plus examples/realphoto_nerve_vs_heart_* and examples/noisy_nerve_vs_heart_*.

Repository layout

nerve_arch.py                    the architecture (copy to traiNNer/archs/)
icnr.py                          ICNR init helper (copy to traiNNer/utils/; not in upstream yet)
configs/                         example training configs (+ 4x_NERVE_onnx.yml for ONNX export)
models/                          pretrained checkpoints (see table above)
onnx/                            dynamic fp32 ONNX (opset 20, onnxslim-optimized)
ncnn/                            ncnn fp16 param/bin
docs/ABLATIONS.md                the full development/ablation log (what we tested)
examples/                        real-image comparisons (input vs bicubic vs release vs GAN)
scripts/export_dynamic.py        checkpoint → dynamic ONNX exporter (+ ORT verify)
LICENSE                          Apache-2.0

Help wanted (community welcome)

The goal here is real-world usability — easy training, hassle-free dynamic ONNX, simple and robust to maintain — not leaderboard chasing. So NERVE's strength is simplicity for trainers; there's still plenty worth measuring. Pick anything below and open an issue/PR/discussion with your numbers.

Benchmarks (deliberately not chased here)

Train on an academic dataset (DIV2K / Flickr2K / LSDIR) and compare against SPAN, SRVGGNetCompact, HAT, Real-ESRGAN (PSNR / SSIM / LPIPS).
Perceptual metrics (LPIPS / TopIQ / MUSIQ) on real-world degraded images.

Speed & deployment (not benchmarked here)

TensorRT / DirectML latency + peak VRAM on the dynamic ONNX.
Static-shape ONNX (e.g. 1x3x256x256) vs dynamic: speed + memory.
fp16 vs fp32 ONNX.
ncnn on-device (Android / iOS): latency + memory.
CoreML conversion + on-device test.
Mobile CPU (XNNPACK / ORT Mobile / ncnn).

Models & code

More finetunes (illustration, manga, denoise, 1x restoration).
Architecture experiments — see docs/ABLATIONS.md for what we already tested (and rejected).

Small, measured contributions are welcome.

License

Apache-2.0 (this repository's code and pretrained weights).

Downloads last month: -; Downloads are not tracked for this model. How to track

Phips
/

NERVE