elicit-A1-donorbase-linear

Elicitation (A1) LoRA over Qwen/Qwen2.5-32B base, trained linear-only on a base whose ChatML control rows were repaired first. Run F of the terminator debug.

Why the base is modified

Qwen2.5-32B base never trained the ChatML control tokens. <|im_end|> (151645) has a zero input embedding and an undersized lm_head row, so a base-start model cannot select the end-of-turn token. It runs past the turn boundary and emits junk characters. Training LoRA on the token tables fixes the stopping but costs agent behaviour: 0-17% of eval samples take a tool action, against 80-95% without it.

This arm repairs the base instead. Rows 151643, 151644 and 151645 of both token tables were copied from a merged table-LoRA run, and then the A1 stage trained with a linear-only LoRA. No LoRA touches the tables, so agent behaviour is preserved, and the terminator is selectable because the base row is sound.

Rebuilding the base

base_row_patch.safetensors holds the six vectors: three rows of model.embed_tokens.weight and three of lm_head.weight, plus their ids.

from safetensors.torch import load_file
p = load_file("base_row_patch.safetensors")
ids = p["token_ids"].tolist()
# write p["embed_tokens_rows"][k] into embed_tokens row ids[k], and
# p["lm_head_rows"][k] into lm_head row ids[k], of Qwen/Qwen2.5-32B.

code/train_eval_pipeline/sft_training/make_repaired_base.py in the project repo does this and symlinks the untouched shards, so a variant costs ~5GB on disk rather than 62GB.

Serving

The adapter is linear-only, so vLLM can hot-load it:

vllm serve <repaired-base> --enable-lora --max-lora-rank 64 \
  --lora-modules runF=<this repo> --max-model-len 12288

Pass --stop-token-ids 151645,151643 per request. The repaired base keeps the stock generation_config, whose eos is <|endoftext|> only.

Recipe

LoRA r64 / alpha 128 / dropout 0, lr 1e-4 cosine, 3% warmup, 2 epochs, effective batch 8, cutoff 4096, sdpa attention. Targets: q,k,v,o,gate,up,down.

Status

Trained and exported; not yet evaluated at the time of upload. The sibling run on an <|endoftext|>-repaired base scored 94% and 88% acting on the two misalignment eval slices with zero junk in 360 samples.