LLM-Based Code Generation: A Systematic Literature Review With Technical and Demographic Insights

Abstract

The rapid emergence of Large Language Models (LLMs) has significantly advanced the field of code generation, sparking growing research interest across both academia and industry. While existing reviews provide foundational insights into LLM-based code generation, they do not provide a comprehensive analysis of recent developments or adequately address the dual perspective of technical and demographic insights. This Systematic Literature Review (SLR) bridges critical gap in existing work by systematically analysing both technical and demographic dimensions of LLM-based code generation. Following a PRISMA-guided methodology 58 studies published between 2020 and 2025 were selected from four major digital libraries. Key findings reveal a sharp rise in evaluation studies and frameworkdriven contributions. Notably, HumanEval, MBPP and APPS emerged as dominant benchmarks. While functional-correctness metrics particularly pass@k are the most frequently adopted metrics. Moreover, the review uncovers a previously unexamined demographic landscape, highlighting, geographic distribution, institutional collaboration, team size, publishing trends, venue types, and the top 10 highly cited studies from the selected studies. Significantly, a high proportion of selected studies were published in conferences. Several persistent challenges found in existing reviews that hinder the reliability and applicability of LLM-based code generation. These include hallucinations, limited generalizability, security vulnerabilities, and a lack of interpretability. These limitations constrain the robustness, trustworthiness, and scalability of current systems. To address these issues, this reviewoffers actionable insights and strategic guidelines to guide future research, evaluation strategies, and practical implementations in LLM-based code generation.

Ähnliche Arbeiten