OpenAI Parameter Golf — What It Taught Us

TL;DR

OpenAI ran a community ML challenge (16 MB model, 10 min training, 8×H100s). Key finding: AI coding agents have become standard research tooling, dramatically lowering the barrier to entry for ML research.

Authors & Org

OpenAI (research blog, not a peer-reviewed paper). Community challenge: 1,000+ external participants, 2,000+ submissions.

Method

• Constraint: Model + training code ≤ 16 MB artifact; training ≤ 10 minutes on 8×H100s
• Evaluation: Language model quality within constraints
• Community: Open challenge via github.com/openai/parameter-golf; $1M compute credits from RunPod

Results

1. Competitive LMs achievable in 16 MB — efficiency research has headroom beyond scale.
2. Winning approaches: aggressive quantization + unconventional architecture design.
3. Coding agents cited by majority of participants as core research workflow — experiment setup, unfamiliar code inspection, hypothesis testing. First major ML competition where agent-assisted research was normative.

Significance

Two distinct signals:

Efficiency signal: Capability exists independently of raw compute scale. Relevant to edge/mobile deployment, cost reduction, and regulatory environments that restrict large clusters.

Agent-assisted research signal: The widespread agent use is a direct data point on the Software 3.0 Verifiability Principle: RL agents thrive when a clear, auto-evaluable reward signal exists (here: validation loss + model size). ML research — with its inherently quantifiable objectives — may be one of the highest-ROI domains for agentic automation. Connects to Andrej Karpathy's autoresearch project.

Open Questions

• Do agent-assisted approaches generalize beyond tightly constrained challenges to open-ended research?
• Does 16 MB efficiency transfer to production-scale model design decisions?

Cite

"What Parameter Golf taught us" — OpenAI research blog, 2026-05-12. (link)