Build a "Jev" From Scratch — toy System One model
⚠️ IMPORTANT — honest framing. This is a toy-scale reconstruction of the System One
model interface that TypeSafe AI's Jev demonstrated (announced Sep 15, 2026). The
real Jev's internals are proprietary and unpublished. This model is NOT Jev and does
not claim to be. The architecture, heads, losses, and calibration here are our own design
that reproduces Jev's proven interface (state + typed questions → calibrated parallel
probabilities). See ARTICLE.md for the full honest story with sources.
What this model does
Given one state (text) and several typed questions, it answers them in parallel
(one encoder pass over the state), no text generation:
- noul → probability a yes/no statement is true
- choice → probability distribution over options + confidence
- score → a value in a range
Real results (trained on CPU, 8-core, 15 GB RAM)
| Type | Dataset | Acc | Brier | ECE |
|---|---|---|---|---|
| noul | BoolQ + SST-2 | 59.7% | 0.236 | 0.027 |
| choice | AG News | 75.9% | 0.331 | — |
Training loss 1.55 → 1.00 (3 epochs). ~3.1M params. Low accuracy is expected (toy, tiny
data slice, CPU-only); the calibration (ECE ≈ 0.027) is the architecturally meaningful
result. Full details: RESULTS.md.
Files
ARTICLE.md— the full "let's build a Jev from scratch" article (simple English)RESULTS.md— real training + eval transcriptREADME.md— setup + usagejev_toy/— model, data, train, eval, serve source (PyTorch)checkpoints/model.pt— the trained checkpoint (cfg + state_dict + vocab)
Usage (inference)
import torch
from jev_toy.model import SystemOneConfig, SystemOneModel
from huggingface_hub import hf_hub_download
import pickle
# load checkpoint
p = hf_hub_download("azharmo/build-jev-from-scratch", "checkpoints/model.pt")
ck = torch.load(p, map_location="cpu")
cfg = SystemOneConfig(**ck["config"])
model = SystemOneModel(cfg); model.load_state_dict(ck["state_dict"]); model.eval()
See jev_toy/serve.py for a full Jev-shaped serving example.
Reproduce
python -m jev_toy.train --epochs 3 --agnews 1500 --boolq 1500 --sst2 1500
python -m jev_toy.eval --ckpt checkpoints/model.pt
python -m jev_toy.serve --ckpt checkpoints/model.pt
Sources
- Jev / TypeSafe: https://typesafe.ai/blog/introducing-system-one-models-and-jev
- Needle / Cactus (fully open): https://github.com/cactus-compute/needle
Educational reconstruction. Not affiliated with TypeSafe AI or Cactus Compute.