$ cat wiki/papers/2026/2607.14777-seed-agentic-rl-distillation.md
SEED: Self-Evolving On-Policy Distillation for Agentic Reinforcement Learning
TL;DR
SEED converts an LLM agent's own completed trajectories into natural-language "hindsight skills," then distills those skills back into the policy during RL training — bootstrapping agentic capability from sparse rewards without human demonstrations.
Authors & Org
Yuhao Shen, Fan Zhang, Zhengxi Lu, Shuo Yang, Jinyang Wu — Tsinghua University + Zhejiang University. Built on the veRL/verl-agent framework.
Method
Three-stage loop, all running on the same policy model:
- Trajectory harvesting: rollout the current policy on agentic tasks with sparse terminal reward
- Skill synthesis: an LLM-as-critic pass converts each trajectory into structured natural-language skills — reusable sub-workflows, decisive observations, failure-avoidance rules
- On-policy distillation: synthesized skills injected as behavioral priors for the next rollout batch
No expert demonstrations. No off-policy data. The policy bootstraps from its own experience.
Results
Outperforms baselines on agentic task benchmarks. Specific numbers not yet confirmed; 86 upvotes on HuggingFace Daily Papers (July 17) reflect strong community signal. (source)
Significance
Addresses the cold-start problem in agentic RL: the model needs no prior expert data — it synthesizes skills from whatever it managed to do, then builds on them. The hindsight-skill representation is human-readable (inspectable), which distinguishes this from pure reward shaping approaches. Directly relevant to multi-step credit assignment, the core unsolved problem in Agentic Reinforcement Learning.
Open Questions
- Does the hindsight skill quality plateau as the policy improves, or does the loop compound?
- Tested at what scale? Does the approach transfer to frontier-size models?
Cite
arXiv:2607.14777 (2026-07-17). HuggingFace Daily Papers July 17, 2026.
Related
- Agentic Reinforcement Learning — the field this addresses directly
- Agents (LLM Agents) — the LLM-as-agent paradigm
- Weak-to-Strong Generalization via Direct On-Policy Distillation — related distillation work (same week)
- The Mirage of Optimizing Training Policies: Monotonic Inference Policies as the Real Objective for LLM Reinforcement Learning — RL training-inference mismatch (MIPI; SEED adds skill-based correction)