GRPO / GCPO / DAPO / AGRO checkpoints

This public repository contains 15 merged bfloat16 Hugging Face checkpoints from 9 reinforcement-learning experiments based on Qwen/Qwen3-1.7B-Base (base revision ea980cb0a6c2ae4b936e82123acc929f1cec04c1).

Layout

Experiment	Repository subfolder	Steps
AGRO SeqSum, beta=0.001	agro-seqsum-beta0.001/step-{N}	100
GCPO Exp SeqMean, beta=0.001	gcpo-exp-seqmean-beta0.001/step-{N}	360, 500
GCPO Exp SeqMean, beta=0.01	gcpo-exp-seqmean-beta0.01/step-{N}	340, 500
GCPO Exp SeqMean, beta=0.1	gcpo-exp-seqmean-beta0.1/step-{N}	500
GCPO Exp SeqMean, beta=1	gcpo-exp-seqmean-beta1/step-{N}	500
GCPO Exp SeqMean, beta=1, plain shuffle	gcpo-exp-seqmean-beta1-plain-shuffle/step-{N}	440, 500
GRPO, KL=0.001	grpo-kl0.001/step-{N}	420, 500
Matched DAPO	matched-dapo/step-{N}	220, 300
Matched DAPO + Token-TIS	matched-dapo-tis/step-{N}	220, 300

Each subfolder is a standalone Transformers model containing merged model.safetensors, model configuration, and tokenizer files.

Checkpoint pruning (2026-08-25)

Intermediate checkpoints were pruned; each experiment now keeps its peak checkpoint (selected by the training-time MATH500 validation curve) and/or its last checkpoint. The per-step MATH500 validation history for every experiment remains fully recorded in W&B.

Loading

from transformers import AutoModelForCausalLM, AutoTokenizer

repo_id = "Thunderous77/grpo"
subfolder = "gcpo-exp-seqmean-beta1-plain-shuffle/step-500"

tokenizer = AutoTokenizer.from_pretrained(repo_id, subfolder=subfolder)
model = AutoModelForCausalLM.from_pretrained(
    repo_id,
    subfolder=subfolder,
    dtype="auto",
    device_map="auto",
)

Important scope

These uploads are standard merged model weights intended for inference, evaluation, or further initialization. Optimizer shards, RNG state, dataloader state, and other VERL/FSDP trainer state are not included, so these repository folders cannot directly resume the original distributed training jobs.

Before local cleanup, every source checkpoint was checked for complete 8-way model/optimizer/extra-state shards. Every merged remote model.safetensors was then verified against its local SHA-256 digest.