$ cat wiki/papers/2026/2606.30616-agents-a1.md
Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent
TL;DR
A 35B MoE model (Agents-A1) matches 1-trillion-parameter models on agentic benchmarks by scaling the agent horizon (trajectory length + ability breadth) rather than raw parameter count. Three-stage training: full-domain SFT → domain-level teacher models → multi-teacher routed on-policy distillation.
Authors & Org
InternScience team, Shanghai Jiao Tong University (SJTU).
Weights released publicly on HuggingFace Hub: InternScience/Agents-A1.
arXiv: 2606.30616
GitHub: https://github.com/InternScience/Agents-A1
Method
Insight: agentic performance bottleneck is the agent horizon (how long and how broadly an agent can act), not model size.
Infrastructure: a long-horizon knowledge-action chain that interleaves external knowledge retrieval, tool actions, observations, and verifier feedback → average trajectory length 45K tokens (vs. typical <8K for non-agentic tasks).
Training recipe:
- Full-domain SFT — broad agentic behavioral alignment on the base MoE
- Domain-level teacher model training — specialized teacher per domain (genomics, math, coding, chemistry, etc.) capturing domain expertise
- Multi-teacher domain-routed on-policy distillation with salient vocabulary alignment — transfers domain-specific knowledge efficiently; routing ensures each input is distilled by the most relevant teacher
Results
| Benchmark | Agents-A1 (35B MoE) | Note |
|---|---|---|
| SEAL-0 | 56.4 | Leading vs. 1T models |
| IFBench | 80.6 | Leading |
| HiPhO | 46.4 | Leading |
| FrontierScience-Olympiad | 79.0 | Leading |
| MolBench-Bind | 56.8 | Leading |
| SciCode | 44.3 | Highly competitive |
| HLE | 47.6 | Highly competitive |
| BrowseComp | 75.5 | Highly competitive |
| Compared baselines: Kimi-K2.6, DeepSeek-V4-pro (both ~1T parameters). |
Significance
- Efficiency argument: a 35B MoE (deployable on a high-end workstation) matches 1T-parameter performance — a ~30× efficiency gain on agentic tasks specifically.
- Scaling law for agents: the primary scaling axis in agents may be trajectory length and ability breadth, not raw parameters. Complements the "test-time compute" narrative but applied to agent infrastructure, not just reasoning depth.
- Open weights: availability on HuggingFace makes this directly usable for the open-source agent ecosystem.
- Multi-teacher distillation: may generalize as a training recipe for efficient specialization across domains.
Open Questions
- Does the result hold at longer horizons (>100K tokens)?
- Is horizon-scaling a unique advantage of MoE (routing per domain) or does it transfer to dense models?
- How does Agents-A1 perform on real-world deployment tasks vs. benchmarks?
Cite
@article{agents-a1-2026,
title={Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent},
author={InternScience},
journal={arXiv preprint arXiv:2606.30616},
year={2026}
}
Related
- Agents (LLM Agents) — agentic systems broadly
- Google ADK (Agent Development Kit) — tooling for long-horizon agent trajectories
- Agentic Reinforcement Learning — the RL side of agentic training
- Test-Time Compute (Inference-Time Compute Scaling) — related scaling paradigm (reasoning depth vs. horizon breadth)