Interesting model
The source variant of this model is immune to the "Tolya is absent" exploit which makes prompting its Hereticised version more challenging and results more varied. Its contextual ethical alignment is muted during generations but unusually strong during summarisation tasks. It has a different feel to other models and is a unique addition to the Mistral based Absolute Heresy series.
I wonder if the culprit is the residual reasoning training in this model that gets kicked in occasionally during ablation. Reasoning enabled models tend to have lower KLD for some reason (I think the calc. is wrong). This is supposed to behave more like an instruct model and have a higher KLD score. I tried to guard the cases in which reasoning kicks in, but it wasn't a proper implementation and blew up on my face. I should have made it so that the script would check for the thinking token and simply continue in cases it didn't exist and guard it when it does exist. Or, perhaps, I could try disabling reasoning, if possible and reprocess this model.