Auron-279M (Archived)

Note: This model is part of a scaling study. The 279M achieved a final val_loss of 3.188 — virtually identical to the 4x larger 1.1B model (3.180), revealing a scaling wall in the Ouroboros weight-sharing mechanism. The 510M model is the best-performing Chimera variant.

For inference and testing, use Auron-510M (val_loss 3.035).

Model	Params	Final Val Loss	Status
Auron-279M	279M	3.188	Archived
Auron-510M	510M	3.035	Best
Auron-1.1B	1.1B	3.180	Archived

Paper: Auron | Code: github.com/Fy-/Auron | Blog: HuggingFace

Architecture

• Type: Chimera (4 bottom + 4×3 top = 16 virtual)
• Dim: 1024, head_dim=64, expand_v=2
• Params: 279M (123M unique + 155M embed)
• Trained: 250K steps, 5B tokens, WSD schedule

from ouro import load_model, generate
model, tokenizer, device = load_model("nyxia/Auron-510M")  # Use 510M
generate(model, tokenizer, device, "The history of")

Built by Florian Gasquez (@nyxia). Part of Soulkyn.