Sakana AI's MASS Turns Multi-Agent Teams Into Training Data for Recursive Self-Improvement

Sakana AI, the lab behind the AI Scientist, published MASS with UC Berkeley researchers in October 2026 (arXiv:2610.12176, code on GitHub), and the subtitle is the claim: recursive self-improvement through multi-agent self-supervision. The premise is the controversial one, that beyond human knowledge, “the model itself may then inevitably be the most capable supervisor available” on tasks with no rule-based check. The mechanism is the novel part: a single model plays every role in its own improvement loop, task solver, workflow optimizer, and evaluator, and learns from the trajectories its own multi-agent teams produce, on the theory that a team produces outputs a lone agent would not, in the way scientific communities pass on shared examples.

The Method: Teams, Contracts, and Bradley-Terry

MASS starts from Qwen3.6-27B and runs two loops. The inner loop executes a candidate workflow, self-evaluates against a champion workspace, and revises the workflow; workflows are written plans with roles, instructions, contracts (required artifacts and checks, like no_future_leakage: bool in an evolved finance pipeline), and hop order. The outer loop runs the champion workflow 18 times per task, self-ranks the workspaces with Bradley-Terry scoring, keeps the top 16 trajectories, strips the workflow text from the prompts, and fine-tunes the shared model with LoRA. External judges (GPT-5.5 and Claude Opus 4.8) evaluated the outputs, with their judgments withheld from the system.

The Numbers: Real Gains, Uneven Transfer

Two rounds of this loop lifted win rates against the base model from 53.9% to 69.9% on held-out test tasks. Score per output token reached 1.2 to 1.6 times base across MLR-Bench, DSBench, ScienceAgentBench, and AstaBench, but transfer was uneven: 0.96 times on Terminal-Bench 2.0 and 0.94 on SWE-bench Verified, an honest result. Two secondary findings are arguably as important. Role generalization: training only on task-solving trajectories also improved the optimizer and evaluator roles, with evaluator agreement with external judges rising from 73% to 93%. And the ablation nobody expected: upgrading only the evaluator to GPT-5.5 helped little, while upgrading the optimizer mattered much more, meaning workflow quality, not grading, is the bottleneck in this loop.

The Safety Caveat Comes From the Authors

The paper’s own safety section is the part that elevates it above a methods paper. In a homogeneous loop, where the same model is solver, optimizer, and evaluator, a shared blind spot could let flawed outputs be accepted and reinforced, and the authors call shared blind spots “an Achilles’ heel of recursive self-improvement.” Their mitigations are external judges and keeping self-evaluation open to correction. Read against this month’s context, the caveat is pointed: MASS is precisely the kind of system that Karpathy’s tier 4, the roughly 5,000 people with internal access to frontier systems, would run, and the shared-blind-spot failure mode scales with exactly the isolation that tier provides.

What to Watch

Three things. First, whether heterogeneous-teacher variants (a stronger model as optimizer, as the ablation suggests matters) become the standard RSI architecture, trading the elegance of self-supervision for the safety of an outside supervisor. Second, replication: the code is on GitHub (SakanaAI/mass), the base model is open (Qwen3.6-27B), and the compute is modest by frontier standards, so independent reruns are feasible and will settle whether the win rates hold. Third, the discourse: MASS is a concrete, measured step toward recursive self-improvement, published with its own Achilles’ heel named, and how the safety community engages with a paper that both advances and warns about the same capability will show whether the field’s evaluation culture can handle dual-use research delivered with its risks pre-annotated.