Shieldstral (arXiv:2607.25857)

TL;DR

A 3B multimodal safety classifier that takes its moderation policy as natural language at inference time instead of predicting fixed harm categories, and returns one continuous score. It reports 84.9% average F1 on text safety and 83.8% on multimodal safety, against models up to 7× its size (source).

Authors & Org

Mistral AI and collaborators. Submitted 2026-07-28; the weights shipped 2026-08-04 as Shieldstral 1.0 (source).

The author list was not obtainable from this environment — arxiv.org returned HTTP 403, as it has on every run this week, and no page of the paper was read directly. Everything below comes from search-result extracts of the arXiv abstract and the Hugging Face paper page, and is recorded with that provenance (source).

Method

• Moderation is formulated as a binary question-answering task — a single yes/no problem rather than a multi-label classification over a fixed taxonomy
• The paper's stated reason for that formulation is a training-data one: it lets heterogeneous safety datasets with divergent taxonomies be consolidated under one framework, which multi-label training cannot do without first reconciling the taxonomies
• The policy is supplied at inference time in natural language; the model returns a single continuous safety score rather than a label
• Data recipe: curation and generation of approximately 54.1M samples, plus a fine-grained evaluation set built specifically to measure policy adaptability — that second artefact is what separates the claim "it can follow a new policy" from "it scores well on the policies it was trained on" (source)

Results

Benchmark set	Average F1
Text safety	84.9%
Multimodal safety	83.8%
Model	Multimodal average F1
---	---
Shieldstral (3B)	83.8%
OmniGuard-7B	77.6%
LlavaGuard-7B	71.6%
All figures are the paper's own
(source). No independent
reproduction was published in any source read.

Significance

The interesting claim is not the parameter count. It is that the taxonomy moved out of the weights and into the prompt. Every guardrail model this wiki has encountered bakes its harm categories in at training time, which means a platform whose acceptable-use policy changes — or which operates under two jurisdictions with different rules — either retrains or accepts a mismatch. Reformulating as binary QA against a supplied policy is what makes that possible, and the taxonomy-consolidation argument suggests the training-data benefit came first and the deployment flexibility fell out of it.

The calibrated continuous score matters for the same reason: a deployer choosing its own threshold is choosing its own false-positive rate, which a discrete label forecloses.

Apache 2.0 at 3B on a 16GB GPU puts a moderation layer inside reach of operators who cannot buy one — see Open-Weights Policy Fight, where the Open Secure AI Alliance's founding argument is precisely that defensive tooling should be open.

Open Questions

• No independent evaluation exists. Both headline figures are the authors' own, and this repo holds no leaderboard snapshot with a safety-classification column to check them against
• Policy adaptability is measured by an evaluation set the same authors built. The artefact is the right one to build; it is not an outside check
• Adversarial robustness is not addressed in anything read. A classifier that reads its policy from the prompt raises the question of what a prompt-injected input does to it, and the abstract does not say
• The 12 languages are not named, nor is per-language performance reported
• Author list unknown — see above

Cite

Shieldstral. arXiv:2607.25857, 2026.

• arXiv:2607.25857
• Hugging Face paper page
• Snapshot: (source)
• Model page: Shieldstral 1.0