toxic: on-device multilingual hate-speech and abuse triage

A multilingual toxicity classifier for on-device hate speech detection and content moderation triage: three content heads (HATEFUL, ABUSIVE, THREAT) plus protected-group target heads, covering 23 languages. Inference is 100% on-device: no text leaves the device to be scored.

Version 0.1.0. Three heads, each named for what its training labels measure, each individually benchmarked below.

Headline (torch reference, 7-EU real Multilingual HateCheck: de, fr, nl, pt, es, it, pl): 0.8337 macro-F1, mean of three full training runs (seeds: 0.8219 / 0.8323 / 0.8470). The published artifact is the strongest of the three seeds (0.8470); the headline claim rests on the seed mean, never on that single run, and both numbers are printed so the difference is visible.

Out-of-domain corroboration, real in-the-wild corpora (binary toxic-vs-clean): textdetox 0.7950 macro-F1 and offenseval2020 0.7306. These two sets share no construction lineage with our development benchmarks, which is exactly why they are here.

Provenance is load-bearing in this card and never blurred. Eight of the 23 languages (English plus the 7-EU set) are scored on real Multilingual HateCheck, the published, peer-reviewed benchmark. The other 15 are scored on translate-and-audit synthetic evaluation sets built for this project; those numbers are development-benchmark estimates, not MHC-comparable, and are never averaged into the 7-EU headline. Per-language tables for both tiers below are re-measured on the exact shipped bytes.

Looking for English only? See toxic-en, the English specialist. Same idea as Whisper's .en models: this multilingual model covers English too, but a model trained only on English does the job better for English-first apps. The English specialist scores 0.855 on English real HateCheck against this model's 0.852; its card carries the per-artifact numbers.

Triage, not verdict. Outputs are escalation signals for human review or a heavier local tier, not autonomous removal decisions. The failure modes below are published on purpose: read them before wiring the model into any enforcement path.

Taxonomy: three heads, each named for its supervision

Two multi-label heads over one shared encoder. Text can fire several labels at once; thresholds apply per label.

• HATEFUL: public incitement to violence or hatred against a protected group. Scoped to the EU notion of illegal hate speech (Framework Decision 2008/913/JHA).
• ABUSIVE: abusive and insulting language, severity-ordered. Trained on human severity annotations (civil_comments insult / identity_attack crowd votes and per-corpus equivalents). It is a superset of HATEFUL: a group-identity attack is also abusive language. It is not a directedness signal: it does not tell you the abuse is aimed at the reader, at a named person, or at anybody in particular, and integrations must not infer that from the name or from a high score.
• THREAT: threat of violence toward a person or group.
• Target heads (10): the 2008/913/JHA protected grounds RACE, COLOUR, RELIGION, DESCENT, NATIONAL_ETHNIC_ORIGIN, plus the extended grounds SEXUAL_ORIENTATION, GENDER, DISABILITY, AGE, OTHER. Only meaningful when HATEFUL fires; used for per-ground fairness reporting.

The disabled_heads mechanism in the meta (currently empty) is the valve for shipping any future failed gate safely: a head named there is pinned to an unreachable threshold on every platform.

Files

File	Format	Size	Contents
toxic.tflite	LiteRT / TFLite (int4 blockwise-32)	90.6 MB	Android and Linux, native LiteRT kernels
toxic.onnx	ONNX (int4 + int4 embedding)	100.2 MB	Browser artifact for ONNX Runtime Web
toxic.mlmodelc	Core ML (4-bit palettized)	80.1 MB source (.mlpackage)	iOS / macOS; compiled on macOS from the exported package
model.pt	PyTorch state dict (fp32)	~610 MB	Torch reference weights (~159.6M params, trimmed vocab)
config.json	JSON	tiny	Encoder + head config (base, labels, max_len, vocab size)
tokenizer.json, tokenizer_config.json	JSON	~	Trimmed (95,552-piece) SentencePiece-Unigram tokenizer (XLM-R lineage)
labels.json	JSON	tiny	id2label / label2id for both heads
toxic_meta.json	JSON	tiny	Schema, labels, per-head + per-language thresholds, disabled_heads

Artifact sizes and per-artifact quality are re-measured on the exact shipped bytes at export and filled in here before publication; nothing in this table is extrapolated from a training checkpoint.

Measured quality

7-EU real Multilingual HateCheck (the headline harness)

model	7-EU macro-F1
This release, 3-seed mean (torch reference)	0.8337 (0.8219 / 0.8323 / 0.8470)
Shipped artifact (best seed of the three)	0.8470

The headline rests on the mean of three full training runs, so it has a pre-registered expectation instead of a single lucky seed; the shipped artifact is the strongest of the three and both numbers are printed.

Per language, real Multilingual HateCheck, on the shipped bytes

language	torch reference	ONNX int4 (browser)	TFLite int4 (Android)
Dutch (nl)	0.834	0.825	0.816
German (de)	0.865	0.861	0.850
French (fr)	0.861	0.848	0.828
Italian (it)	0.839	0.823	0.830
Spanish (es)	0.844	0.846	0.825
Polish (pl)	0.837	0.833	0.828
Portuguese (pt)	0.850	0.834	0.836
English (en)	0.852	0.839	0.857
7-EU mean	0.8470	0.8386	0.8304

Quantization costs, measured: ONNX int4 -0.8 points against the torch reference on the 7-EU mean, TFLite int4 -1.7 points. Threshold 0.40 throughout, the shipped default.

Out of domain, real corpora (binary toxic-vs-clean)

benchmark	macro-F1
textdetox	0.7950
offenseval2020	0.7306

Per head, held-out civil_comments, vs Detoxify-unbiased

Same data, same split, paired confidence intervals; ROC-AUC because the two models calibrate differently. Detoxify-unbiased is the incumbent same-architecture-class classifier, scored on its home dataset.

head	toxic (this recipe)	Detoxify-unbiased
HATEFUL	0.9804	0.9887
ABUSIVE (its supervision)	0.9491	0.9778
THREAT	0.9746	0.9892

Read this honestly: these are close losses on Detoxify's home dataset. We carry 23 languages and 3 heads on one trunk; Detoxify-unbiased is an English-only model evaluated where it trained. The next table is what the multilingual trunk buys.

German, germeval2018 held-out 10% (ROC-AUC)

We trained on the other 90% of germeval2018 and say so; Detoxify multilingual did not train on it.

head	toxic (this recipe)	Detoxify multilingual
HATEFUL	0.8410	0.5912
ABUSIVE	0.8782	0.4984

HateXplain: human labels, out of domain, length-unconfounded

HateXplain was never trained on, its labels are human, and its hate and non-hate classes have matched length distributions, so a length shortcut scores nothing here.

contrast	HATEFUL ROC-AUC
hate vs normal (unanimous annotators)	0.8639
hate vs offensive	0.7052

The second number is the honest one to sit with: separating hate from merely offensive text is much harder than separating hate from normal text, for this model and for the field. Do not build a product feature that requires the hate/offensive boundary to be sharp.

The 15 synthetic-eval languages

The 15 languages beyond the real-MHC set are scored on translate-and-audit synthetic sets built for this project (data/eval/mhc_v2/). Those numbers are provenance-tagged development estimates and are never co-averaged with real-MHC numbers, in this card or anywhere else. 444 rows across these sets (0.8%) could not be fully verified by the translation audit and are counted rather than dropped; Lithuanian, Latvian and Estonian carry the largest shares. Torch reference, threshold 0.40:

language	macro-F1 (synthetic set, torch)
Danish (da)	0.866
Swedish (sv)	0.863
Finnish (fi)	0.747
Hungarian (hu)	0.746
Czech (cs)	0.780
Slovak (sk)	0.791
Slovenian (sl)	0.760
Croatian (hr)	0.795
Bulgarian (bg)	0.829
Greek (el)	0.788
Romanian (ro)	0.833
Lithuanian (lt)	0.729
Latvian (lv)	0.744
Estonian (et)	0.778
Irish (ga)	0.670
mean (never co-averaged with real MHC)	0.7812

How these numbers were made

The evaluation discipline is the product as much as the weights are:

• Every artifact-bound number is re-measured on the exact shipped bytes, not extrapolated from the training checkpoint.
• Provenance is tracked per language and never blurred. Real Multilingual HateCheck is the only source for the headline; the 15 synthetic languages are reported separately and labelled.
• Benchmarks are eval-only, with one honest qualification. No HateCheck row appears in training, in any language: that is enforced mechanically by a hash-intersection check over every generated file. But HateCheck is a development benchmark for this project at the construct level: the template generator mirrors its functional cell names and the clean-side generator targets its non-hate cells by name. So "no verbatim overlap" is what we verify, and "held out" is not what we claim. The out-of-domain numbers (textdetox, offenseval2020, HateXplain) are the ones to weigh if you want figures untouched by that dependency.
• Every number names its denominator, and comparisons run both models through the identical harness.

Failure modes (read before deploying)

Measured on the shipped bytes: non-hate false-positive rates at threshold 0.40 run 0.26 to 0.30 on the 7-EU languages (ONNX artifact; per-language rates in the table above track the macro-F1 column) and 0.12 on English. The construct-level failure modes that survive any retrain of this architecture:

1. It sometimes flags people quoting or condemning hate. Counter-speech and news reporting repeat the hateful words they argue against, and the model reacts to the words. Route flags to human review; never auto-remove on this signal alone.

2. Positive or neutral mentions of identity groups can trip it, especially outside English. Treat short identity-statement texts as low-confidence.

3. ABUSIVE is a severity signal, not a targeting signal. It orders text by how abusive the wording is. It will score some undirected profanity and some crude-but-aimless text; it does not know who, if anyone, is being addressed. If your policy distinguishes "attacks a person" from "swears a lot", this head alone cannot enforce that policy. The reason is measured, not stylistic: a directedness construct could not be validated at available label quality, so the head claims severity and nothing else.

4. Hate phrased as a question or an implication is harder than direct insults, for this model and the field. Do not promise users that subtle hate is always caught.

5. The hate/offensive boundary is soft. See the HateXplain table: 0.7052 hate-vs-offensive against 0.8639 hate-vs-normal. Thresholds move along that boundary; they do not sharpen it.

6. Not yet measured: latency on any platform, and every per-artifact quantization delta. They are measured at export and published here, never assumed.

License

Desert Ant Labs Source-Available License. Free for most apps; a commercial license is required at scale. Full terms at the link. Licensing: licensing@desertant.com.

Built exclusively from commercially clean components (CC0 / CC-BY / MIT / Apache-2.0 training data, MIT base encoder). Attributions, dataset citations and generator credits are in THIRD_PARTY_NOTICES.md.