Current Large Language Models (LLMs) suffer from significant shortcomings: they are inherently static, unable to adapt internal parameters based on new tasks, evolving knowledge domains, or dynamic interactive environments.
Today, as LLMs are increasingly deployed in open, interactive environments, this static flaw becomes more pronounced, creating an urgent need for agents capable of adaptive reasoning, action, and evolution in real-time—hence, “self-evolving agents.”
Recently, Professor Wang Mengdi’s team at Princeton University released the first systematic and comprehensive review focusing on “self-evolving agents.”
Paper Link: https://arxiv.org/abs/2507.21046
Key contributions are as follows:
Established a unified theoretical framework to describe the self-evolution process in agent systems, structured around “What to Evolve,” “How to Evolve,” and “When to Evolve,” providing clear design guidelines for future self-evolving agent systems;
Examined evaluation benchmarks and environments designed for self-evolving agents, highlighting emergent metrics and challenges related to adaptability, robustness, and real-world complexity;
Showcased key real-world applications in various domains (e.g., autonomous software engineering, personalized education, healthcare, and intelligent virtual assistants), demonstrating the practical potential of self-evolving agents;
Identified critical open challenges and promising future research directions, emphasizing safety, personalization, multi-agent co-evolution, and scalability.
Figure | Evolution panorama of representative autonomous evolving agent frameworks from 2022-2025
By providing a structured framework for understanding and designing self-evolving agents, this review offers a roadmap for advancing adaptive agentic systems in research and practical deployment, pushing towards Artificial Superintelligence (ASI). In this context, agents can not only learn and evolve from experience at unpredictable speeds but also achieve or surpass human intelligence levels across a wide range of tasks.
Current Trends: Self-Evolving Agents
Unlike static LLMs that cannot adapt to new and dynamic interactive environments, self-evolving agents are believed to overcome these shortcomings by continuously learning from real-world feedback.
In this review, the research team analyzed the topic around “What to Evolve,” “When to Evolve,” and “How to Evolve,” and built a structured framework to understand and design self-evolving agents.
Specifically, they systematically studied various components of agents, including models, memory, tools, and their corresponding workflows, and analyzed their evolution mechanisms (“What to Evolve”); subsequently, they classified existing evolution methods according to different time phases and learning paradigms, such as supervised fine-tuning, reinforcement learning, and inference-time evolution (“When to Evolve”); finally, they summarized different evolution signals (e.g., text feedback, scalar rewards) and different agent evolution architectures (e.g., single-agent vs. multi-agent evolution) (“How to Evolve”).
1. What to Evolve?
The self-evolution of agents involves several key components that collectively form the foundation for their adaptation and improvement:
Firstly, the Model, which is the cognitive core of the agent, directly determines its reasoning, planning, and decision-making behaviors. Models optimize their reasoning and decision-making abilities by adjusting internal parameters and learning from their own experiences. These strategies collectively drive a paradigm shift in learning—from passive learning to active, continuous, and self-driven improvement.
Secondly, Context, which includes memory evolution and prompt optimization. Memory evolution focuses on how information is stored, forgotten, and retrieved to aid decision-making, enabling agents to accumulate knowledge, recall past events, and adjust behavior based on experience; prompt optimization enhances model performance by adjusting the phrasing and structure of instructions. Agents can autonomously improve their prompting strategies, transforming prompts into learnable components that evolve with the agent’s experience.
Thirdly, Tool, where agents transform from tool users to creators. This shift from relying on a preset static toolset to achieving autonomous skill expansion and optimization marks a significant leap towards cognitive self-sufficiency. It encompasses the autonomous discovery of tools, mastery through iterative optimization, and efficient management and selection to meet complex task requirements.
Additionally, Architecture is involved. Single-agent system optimization primarily proceeds in two directions: optimizing the agent’s high-level architectural design and enabling it to directly modify its own source code. Performance is enhanced by optimizing nodes and directly integrating component-level optimization into the system architecture search process; complex multi-agent systems, meanwhile, focus on dynamically optimizing collaborative structures to enhance collective problem-solving capabilities.
2. When to Evolve?
The timing of agent evolution is divided into two phases: intra-test-time and inter-test-time, with different manifestations under various learning paradigms. The research team studied these two phases from three dimensions: In-Context Learning, Supervised Fine-Tuning, and Reinforcement Learning:
Intra-test-time self-evolution: Occurs during task execution and is closely coupled with the current task. Through in-context learning, agents adjust their behavior using dynamic memory; supervised fine-tuning enables immediate self-modification; reinforcement learning facilitates targeted learning of new capabilities when challenges arise.
Inter-test-time self-evolution: Occurs after task completion, improving future performance based on historical experience. In-context learning uses past task feedback to assist new tasks; supervised fine-tuning achieves iterative optimization through self-generated data and evaluation; reinforcement learning optimizes strategies by leveraging extensive environmental interactions and curriculum design.
Figure | Overview of reward-based self-evolution strategies
3. How to Evolve?
Figure | Schematic diagram of cross-sectional evolution dimensions during agent autonomous evolution
The ability for self-improvement is the cornerstone of advanced intelligence. In the context of LLMs, this mechanism manifests as a dynamic reward-driven evolution process. Models continuously learn from their own outputs and interactions, progressively enhancing their capabilities. The design of reward signals, which guide the feedback mechanism, is crucial, directly determining the nature, efficiency, and effectiveness of the learning process. The main methodologies for reward design can be categorized into four types based on feedback: text feedback, intrinsic rewards, extrinsic rewards, and implicit rewards.
For more details, please refer to the original review.
Applications: General and Specific Domains
Autonomous evolving agents are poised to drive technological advancements across various domains and application scenarios, primarily involving two broad categories:
General domain evolution: Agent systems evolve to expand their capabilities across a wide range of tasks, primarily focusing on the digital realm;
Specialized domain evolution: Agent systems evolve to enhance their professional capabilities within specific task domains.
Essentially, the evolution of general-purpose assistants focuses on transferring learned experiences to a broader set of tasks, while the evolution of specialized agents emphasizes deepening expertise within a particular domain.
Figure | Evolution directions can be divided into two main categories: general and specialized domains
General domain evolution refers to self-evolving agents designed for general applications, meaning agent systems that expand their diverse task capabilities in the digital domain through evolution. This capability upgrade is primarily achieved in three ways: Memory Mechanisms, Curriculum-Driven Training, and Model-Agent Co-Evolution. These three mechanisms work together to enable intelligent assistants to continuously adapt to complex and changing user needs, providing more efficient service responses.
Specialized domain evolution refers to focusing on enhancing professional skills in specific task domains. In these areas, their evolution is customized to significantly improve performance in narrow task sets, with an emphasis on professional expertise in domains such as coding, GUI, finance, medical, and education. Specifically:
In Coding, self-evolving agents have transformative applications, as their autonomous adaptation and improvement capabilities can enhance software development efficiency and quality. For example, SICA can autonomously edit codebases and improve benchmark task performance; EvoMAC improves code generation by optimizing multi-agent collaborative networks; AgentCoder iteratively optimizes code using a multi-agent framework; and agents continuously evolve by filtering high-quality answers, building machine learning libraries, etc.
In Graphical User Interface (GUI), self-evolving agents extend LLM capabilities from text reasoning to desktop, web, and mobile interface operations, requiring solutions to challenges such as complex action spaces. Related research improves accuracy through pixel-level vision and self-reinforcement; Navi agents improve task completion rates by analyzing failed trajectories; WebVoyager combines screenshots with reflection to increase success rates on unfamiliar websites, with ReAP further improving by adding memory; AutoGUI and MobileUse also enhance capabilities through their respective mechanisms, demonstrating the comprehensive features of self-evolution.
In Financial domain, the bottleneck for customizing agents for professional fields lies in efficiently building and integrating domain knowledge bases, a problem that self-evolution mechanisms can alleviate. QuantAgent iteratively optimizes responses and enhances knowledge bases through a two-layer framework, improving trading performance; TradingAgents integrate various dynamic process optimization strategies.
In Medical domain, self-evolving agents can handle clinical complexities, including hospital-scale simulation, multi-agent collaboration, doctor-patient agent dialogue evolution, reinforcement learning-assisted diagnosis and treatment, architecture search optimization processes, and biomedical discovery.
In Education, self-evolving agents are widely applicable. At the learner level, PACE adjusts prompts and questions based on student situations, and MathVC simulates collaborative learning processes; at the teacher level, i-vip's multi-agent team optimizes output in real-time, EduPlanner optimizes lesson plans through adversarial cycles, and SEFL generates examples for fine-tuning feedback models. These agents can dynamically adapt to the needs of teachers and students, enhancing the educational experience.
Beyond the five major areas listed above, self-evolving agents also show certain advantages in other specialized fields, such as academic assistance, gaming tasks, and diplomatic strategies. They demonstrate broad applicability in their respective domains due to characteristics like continuous learning.
Future Directions: Personalization, Generalizability, Safety and Control
Deploying personalized agents is an important research goal. In applications such as chatbots and digital twins, AI needs to accurately capture and adapt to unique user behavior patterns or preferences. Existing methods rely on labeled data and post-training, but practical challenges include the cold-start problem—how to refine personalized understanding, interpret user intent, and build user profiles when initial data is limited. Additionally, in personalized planning and execution, challenges exist in long-term memory management, external tool integration adaptation, and the reliability of personalized generation, and care must be taken to avoid reinforcing existing biases.
In terms of evaluation, teams need to further break through traditional frameworks and develop lighter, more adaptive metrics, establishing flexible and dynamic benchmark systems to accurately assess agent performance in managing long-tail personalized data during self-evolution.
At the same time, self-evolving agents also face challenges in robust generalization across task domains and environments. The contradiction between specialization and broad adaptability affects system scalability, knowledge transfer, and collaborative intelligence. Scalable architecture design requires building architectures that can maintain performance as complexity and scenarios expand, but current systems often face trade-offs, and increasing dynamic inference computational costs limit generalization capabilities.
In continuous learning, the phenomenon of catastrophic forgetting exacerbates challenges, and balancing efficiency with preventing model drift remains difficult. Knowledge transfer has shortcomings, requiring an understanding of the conditions for knowledge generalization and transfer, quantification of transfer limitations, and establishment of mechanisms that promote the construction of robust world models to improve collaborative efficiency.
Furthermore, as the capabilities of autonomous AI agents strengthen, deploying safer and more controllable agents becomes a key research focus. Current agents still struggle to accurately distinguish necessary sensitive information from irrelevant information, and managing behavior becomes more difficult when goals involve inappropriate means. Learning uncertainty, semantic ambiguity, and design flaws in memory modules all exacerbate safety challenges.
By collecting large-scale, diverse real-world scenario data to support safe behavior learning, refining agent architecture rules and case libraries, exploring safer training algorithms, and investigating the impact of privacy protection measures on agent efficiency, it may be possible to achieve balanced and safe deployment.
Finally, the challenges faced by multi-agent self-evolution systems require them to balance individual and collective reasoning. Research indicates that while collective discussion can improve diagnostic reasoning, agents tend to over-rely on consensus, weakening independent reasoning abilities.
In the future, research teams need to continue exploring dynamic mechanisms to adjust the weights of individual and collective opinions, avoid decisions being dominated by a few, establish explicit knowledge bases and standardized update mechanisms, and enhance individual reasoning contributions in collaboration. Moreover, most existing multi-agent evaluation benchmarks are static, making it difficult to capture long-term adaptability and evolution of roles. Therefore, efficient algorithms and adaptive frameworks need to be developed to enable agents to collaborate effectively while maintaining their own decision-making advantages.
The research team stated that the emergence of self-evolving agents marks a paradigm shift in the field of AI, moving from static, monolithic models to dynamic intelligent systems with continuous learning and adaptive capabilities. As language agents become widely used in open, interactive environments, the key to building a new generation of intelligent systems lies in enabling their reasoning processes, tools, and behaviors to evolve and adapt based on new tasks, knowledge, and feedback.
Looking ahead, fully realizing the potential of self-evolving agents is crucial for building artificial superintelligence, which requires significant breakthroughs in models, data, algorithms, and evaluation. Addressing issues such as catastrophic forgetting, achieving human preference alignment in autonomous evolution, and co-evolution of agents and their environments are key to developing agents that are adaptive, reliable, and aligned with human values.