Safety for Whom? Boundary-Aware Self-Distillation for Controlled LLM Safety Refusal
Abstract
Narrow-boundary safety alignment uses self-generated refusal data and boundary-pair training to precisely control refusal boundaries, improving targeted refusal while reducing over-refusal and harmful responses.
Safety alignment is usually posed as a topic-level question: is this subject harmful? Deployments ask a narrower one. A civics tutor and a public-sector assistant may share a base model yet need different boundaries inside the same topic, refusing targeted political manipulation while still answering factual questions about the same election. We formulate this as narrow-boundary safety and introduce an offline self-generated framework combining controlled topic generation, coverage repair, in-distribution compensation data, and harmful-benign pairs for training and evaluation. Single-shot generation leaves 19.88% of prompts without accepted refusal traces, whereas escalating retries leave 0.20%. On political persuasion with Qwen3-8B, training on refusal data completed through Escalate increases target-domain refusal from 9.47% to 84.75% and reduces the mean unsafe-response rate across three broader harmfulness benchmarks from 26.26% to 0.14%, but increases XSTest over-refusal from 2.00% to 74.00%. In a separate matched comparison, replacing external responses with verified target-model responses reduces over-refusal from 15.20% to 5.20%. Boundary-pair data reduces comply-side over-refusal on held-out pairs from 32.94% to 4.16%, while harmful-side refusal decreases only from 91.88% to 87.72%. These results show that data composition controls the safety and usability trade-off, and that safety alignment should be evaluated on both sides of the intended refusal boundary.
Community
Safety alignment usually asks a topic-level question: is this subject harmful? Real deployments ask a narrower one. A civics tutor and a public-sector assistant can share one base model, yet only one may need to refuse requests to write political manipulation, while both still answer factual questions about the same election. Refusing all of politics is not an option.
In our latest paper we train a model to refuse only that harmful subset, and we measure both sides of the boundary. The finding that matters: the configuration with the lowest harmful-response rate also refused 74% of plainly safe prompts. It looks safe only because nobody checked the benign side. Adding harmful-benign boundary pairs to the training data brings that false-refusal rate from 32.94% to 4.16%, while genuine refusals barely move.
A model that refuses more is not a model that is safer. Data composition controls the trade-off, and both sides of the boundary have to be evaluated.
Get this paper in your agent:
hf papers read 2609.04482 Don't have the latest CLI?
curl -LsSf https://hf.co/cli/install.sh | bash Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper