Unit testing is a critical yet tedious technique for verifying software functionality and mitigating regression risks. Although classical automated methods can effectively explore program structures, they often lack the semantic information required to generate realistic inputs and assertions. Large language models (LLMs) leverage data-driven knowledge of code semantics and programming patterns to address this limitation.
To systematically analyze the latest technological advances in this field, we conducted a systematic literature review of 115 related papers published from May 2021 to August 2025. We proposed a unified taxonomy based on the lifecycle of unit test generation, viewing LLMs as stochastic generators requiring systematic engineering constraints. This framework analyzes existing research in terms of core generation strategies and a series of augmentation techniques, encompassing context reinforcement before generation to quality assurance after generation.
Our analysis reveals that prompt engineering has become the dominant usage strategy, accounting for 89%, attributed to its high flexibility. We found that iterative verification and repair loops have become the standard mechanism for ensuring the reliable usability of generated tests, delivering significant improvements in compilation and execution pass rates.
However, key challenges remain unresolved, including the lack of effective defect detection capabilities in generated tests and the absence of standardized evaluation benchmarks. To propel the field forward, we propose a future research roadmap emphasizing evolution toward autonomous testing agents and hybrid systems integrating LLMs with traditional software engineering tools. This survey offers researchers and practitioners a comprehensive perspective on transforming LLM potential into industrial-grade testing solutions.
Additional keywords: unit testing, automated test generation, large language models
1 Introduction
Software testing is a fundamental engineering practice for ensuring software quality and reducing release risks [98, 109, 128]. As a form of white-box testing, unit testing focuses on verifying the behavior of the smallest independently testable units in the system (such as functions or classes) [15]. Well-designed unit test suites can detect logical errors and boundary condition defects early in development, prevent regressions as software evolves [159], and support agile practices like test-driven development (TDD) [15]. However, manually crafting comprehensive and high-quality unit tests is widely considered a costly and labor-intensive task, reportedly consuming over 15% of development time [31, 117].
To tackle this challenge, automated test generation (ATG) has been a key research focus in software engineering for decades. The field has historically been led by search-based software testing (SBST) [43, 44, 64, 90, 106, 136] and symbolic/concolic execution techniques [26, 50, 132, 160]. These techniques have proven highly effective in systematically exploring program structures. Nevertheless, traditional methods are primarily structure-driven and typically lack semantic understanding capabilities [13, 35, 43, 60, 94, 169]. Unable to comprehend code semantics, they struggle to generate domain-specific formatted inputs, construct objects with complex internal states, or handle interactions with external dependencies (like file systems or network APIs) [45, 113]. Consequently, traditional techniques often fail to produce realistic and effective test cases.
Large language models (LLMs) offer a novel approach to bridge this "semantic gap." Pre-trained on massive code and natural language corpora, these models acquire programming syntax, common patterns, API usage, and domain-specific knowledge [25, 85]. This data-driven foundation empowers them to address semantic challenges that elude traditional methods. They can generate complex inputs imbued with domain semantics [59, 67], create effective test prefixes [108, 170], and produce reasonable mock implementations for external dependencies [54, 108, 116].
The advent of LLMs has accelerated growth in unit test generation research, introducing new challenges and methodological diversity. While prior surveys review LLM applications in software engineering from broader angles [40, 166] or offer preliminary task classifications [165], they primarily collate existing works. This paper posits that LLM-based test generation development should adapt and augment classic software engineering (SE) principles, treating LLMs as stochastic generators needing systematic constraints.
Through systematic analysis of 115 papers, this paper introduces this technical classification framework. Our findings indicate that the research community has swiftly integrated classic SE techniques with LLMs. Notably, program analysis-based context augmentation and feedback loop-centric post-generation repair techniques have emerged as standard practices for enhancing LLM-generated test quality. Advanced efforts are constructing autonomous testing agents and hybrid systems deeply fused with traditional tools. Yet, critical gaps persist: while test usability (compilability and executability) has improved, boosting test effectiveness (real defect detection) remains a formidable challenge. Moreover, absent standardized benchmarks, shallow insight into model limitations, and the academia-industry divide constitute three major hurdles to further progress.
To construct a clear analytical framework, we formulate the following research questions (RQs):
RQ1: How are core LLM usage strategies (pre-training, fine-tuning, and prompt engineering) applied and evolved, particularly their performance in unit test generation?
RQ2: How do classic software engineering techniques augment LLM-based unit test generation, and how do they integrate to form systematic generation process management?
RQ3: What principal challenges confront state-of-the-art LLM-based unit test generation? What opportunities do these challenges afford for future research?
By addressing these questions, this paper delivers a structured overview of the current research landscape and proposes a definitive development roadmap. Grounded in systems engineering principles, it seeks to propel LLM-based testing from academic inquiry to robust industrial application.
To systematically present our analysis, the paper structure proceeds as follows: Section 2 establishes the foundation for our thesis by reviewing classic automated unit test generation principles, delineating the "semantic gap" traditional methods cannot surmount, and elucidating why LLMs are pivotal. Section 3 details the systematic literature review methodology. The core sections address the RQs: Section 4 dissects LLM usage strategies (RQ1); Section 5 constructs the SE techniques ecosystem augmenting and constraining the generation engine (RQ2); Section 6 broadly discusses challenges and pinpoints future opportunities (RQ3). Section 7 addresses threats to validity, Section 8 contrasts related work, and Section 9 concludes, reaffirming core insights on field evolution.