Attributing ChatGPT-Transformed Synthetic Code

Abstract

In this paper, we investigated ChatGPT’s code transformation capability and the effectiveness of the code authorship attribution technique specially designed for ChatGPT code. Through our experiments, we made several key observations. Firstly, ChatGPT demonstrated the capability to transform code in ways that can mislead existing authorship attribution techniques by generating various styles, while it has some constraints, such as the maximum of 12 styles, with certain styles being more commonly employed than others. We also found that the feature-based code authorship attribution proved to be effective when even applied to ChatGPT-transformed code, while the naive approach encountered challenges with accurate classification. In addition, an authorship model trained for binary classification is still effective for ChatGPT-transformed code by achieving up to 93% accuracy. These findings provide insights into the code transformation ability of ChatGPT and shed light on the effectiveness of code authorship attribution techniques for ChatGPT-transformed code.

Ähnliche Arbeiten