Sakana AI's self-improving model gets better at solving and judging, but can't catch its own blind spots

Sakana AI has introduced the MASS (Multi-Agent Self-Improvement) method, which allows a single language model to improve itself by cycling through three roles: solving tasks, judging the results, and refining the workflow. After just two rounds of this iterative process, the model achieved a win rate of nearly 70 percent against its original base version.

The approach is notable because it does not require training a separate judge or reward model. Instead, the same LLM plays all three roles, making the self-improvement loop lightweight and self-contained. This could lower the barrier for teams that lack the resources to build and maintain distinct evaluation pipelines.

However, the method has a key limitation. When the model makes the same errors across all three roles, it cannot identify its own blind spots, leading to persistent mistakes that go uncorrected. This suggests that self-improvement loops are most effective when the model's weaknesses are not uniformly distributed across its different functional roles.

The work contributes to a growing body of research on self-improving AI systems, where models refine their own outputs without external supervision. While promising, the blind-spot problem highlights that fully autonomous self-improvement remains an open challenge, and human oversight or diverse evaluation signals may still be necessary for robust progress.

This page provides an editorial summary based on publicly available information. It is not a republished article. Use the source link below for the original report.