GRAM — Gradient-Routed Auxiliary Modules

Definition

GRAM (Gradient-Routed Auxiliary Modules) is a modular pretraining architecture developed by Anthropic that isolates dual-use knowledge into physically removable compartments within a neural network. During pretraining, gradient routing causes dual-use knowledge (cybersecurity, virology, nuclear physics) to flow preferentially into designated auxiliary module groups. At deployment time, module groups can be physically removed from the checkpoint or selectively disabled at inference without affecting overall model performance.

Why It Matters

GRAM is the first published technique for access-controlled deployment of a single model checkpoint. Prior to GRAM, a lab wanting to deploy different capability profiles to different trust tiers had two bad options: (a) train entirely separate models on filtered datasets (expensive, doubles the compute cost), or (b) apply post-hoc capability elicitation restrictions (fragile; capabilities can often be elicited via prompting).

GRAM makes a "conscience circuit" architecture possible:

• A single training run produces multiple deployment configurations
• The restricted configuration is structurally incapable, not just policy-suppressed
• A dual-use module removal cannot be reversed by prompting, fine-tuning on the deployment checkpoint, or system-prompt injection

This has direct alignment implications: structural incapability of dangerous behaviors is a stronger guarantee than behavioral suppression.

State of the Art (2026-07-21)

Anthropic: GRAM (July 8, 2026) [

Published July 8, 2026 on the Anthropic Alignment Science Blog. (source)

Technical method:

• Extra neurons added to every Transformer layer, divided into groups ("modules") — one per dual-use category
• Gradient routing during pretraining: dual-use knowledge flows preferentially into designated modules
• At deployment: module groups physically removed from checkpoint (restricted tier) or kept (trusted researcher tier)

Test results (5B-parameter model):

• Removing cybersecurity, virology, and nuclear physics modules disabled only those capabilities
• Cross-task benchmarks: not significantly affected (overall performance preserved)
• Tested dual-use categories: cybersecurity exploitation, virology gain-of-function, nuclear physics

Deployment model:

• Full model (all modules on): trusted researchers, controlled environments
• Restricted deployment (dual-use modules removed): public API access
• The same training run produces both configurations — a single checkpoint diverges at deployment

Why it matters for alignment: GRAM is not a behavioral control (RLHF, Constitutional AI, refusal classifiers) — it is a structural control. The restricted checkpoint is literally incapable of the excised knowledge domains, not merely reluctant to use them. This distinction matters as model capability increases: a sufficiently capable model with behavioral suppression may still find ways to surface suppressed knowledge; a model with excised modules cannot.

Connection to existing alignment stack:

• Complements capability gating (Fable 5 safety classifiers, Mythos 5 Glasswing-only access): behavioral gating at inference vs. structural gating at checkpoint level
• Complements Project Glasswing (trusted-partner dual-use access): GRAM provides the technical mechanism for "same weights, different capabilities" — the full-module version goes to Glasswing, restricted version goes to public API
• Complements J-space interpretability (July 2026): J-space detects alignment failures at inference; GRAM prevents structural capacity for them at the weight level

Open Problems

1. Scale validation: results only published for a 5B-parameter model. Does gradient routing remain clean at 400B+ (MoE) where routing itself can be complex?
2. Module boundary leakage: do modules develop cross-module dependencies that partially preserve removed capabilities?
3. Capability-fidelity tradeoff: how much capability loss in the full-module model comes from separating knowledge into compartments?
4. Known unknowns: are there dual-use capability domains that emerge at scale but were not anticipated at the module-design stage? GRAM assumes you can enumerate dangerous categories before training.

Key Papers

• Anthropic (2026-07-08): "Modular Pretraining Enables Access Control" — alignment.anthropic.com/2026/modular-pretraining/ / ae.studio PDF
• Companion post: anthropic.com/research/off-switch-dual-use

Related Concepts

• AI Alignment — GRAM is a structural capability control, complementing behavioral alignment methods
• AI-Enabled Cyberattacks — cybersecurity exploitation is the primary dual-use category driving GRAM's design
• Mechanistic Interpretability — J-space detects alignment failures at inference; GRAM prevents structural capacity at weights
• Claude Mythos Preview — Project Glasswing uses a trust-tiered deployment model; GRAM provides the technical mechanism for that architecture