Unlocking AI Potential: An Investigation of Python Coding Capabilities of ChatGPT and Gemini

Abstract

The rapid advancement of artificial intelligence (AI)-based chatbots has expanded their role in programming, specifically in Python coding tasks. Understanding their capabilities in generating quality code is important to optimize their use in software development. This study compares the Python coding competence of two leading AI chatbots, ChatGPT (OpenAI) and Gemini (Google DeepMind). Using Python programming exercises from the PYnative website, both chatbots were tested on 157 coding questions, and their responses were evaluated for correctness and complexity. The Python Concept Extraction and Representation Framework (PyCEFR) tool was used to analyze the generated code, categorizing it into basic, independent, and proficient user levels. Results indicate that ChatGPT achieved a higher success rate (98.7%) than Gemini (94.9%) and demonstrated a slight edge in producing advanced-level code. Both models predominantly generated beginner-level code, with differences in coding styles, where ChatGPT favoured tuple operations and 'range' functions, while Gemini emphasized loop structures and arithmetic assignments. These findings suggest that while both chatbots effectively assist beginners, improvements are needed for generating expert-level code. Future research should focus on improving AI capabilities for advanced programming tasks and evaluating their real-world applicability in software engineering.

Ähnliche Arbeiten